2024-04-25T14:40:13.445 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T14:40:13.445 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} --archive /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511 --owner scheduled_leonidus@teuthology --name leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/orig.config.yaml 2024-04-25T14:40:13.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:40:13.599 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7673511' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8982 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhmrATMvssx5OYIsjI1YlKVPqtp3z85O1VC1IHWKbIF1iB3JhX9OcNDXWsJD82fWvj26hBhKwPh7Jh3/P1pEjQ= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBApkJcznsj8WnsOIJ8SVPpV7JNCtjdQODYCXrZAnLG84xAi1O0ghj/GK+p0bBV5RAYmlVrklAmLd1GB1U//t3RA= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAZA+gtfIGwyGI91oCGb3Byh26KYW3iDJJL9Jmgauyojp5NdDXjBcHsVdIM1RTSBz3p11BZnESuxL4IVXfU8FA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_13:50:46 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T14:40:13.599 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa; will attempt to use it 2024-04-25T14:40:13.600 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks 2024-04-25T14:40:13.600 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T14:40:13.601 INFO:teuthology.task.internal:Checking packages... 2024-04-25T14:40:13.623 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T14:40:13.623 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T14:40:13.623 INFO:teuthology.packaging:ref: None 2024-04-25T14:40:13.623 INFO:teuthology.packaging:tag: None 2024-04-25T14:40:13.623 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T14:40:13.623 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:40:13.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T14:40:13.786 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T14:40:13.788 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T14:40:13.795 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T14:40:13.795 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T14:40:13.870 INFO:teuthology.task.internal:Saving configuration 2024-04-25T14:40:13.891 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T14:40:13.897 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:40:13.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.806273', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhmrATMvssx5OYIsjI1YlKVPqtp3z85O1VC1IHWKbIF1iB3JhX9OcNDXWsJD82fWvj26hBhKwPh7Jh3/P1pEjQ='} 2024-04-25T14:40:13.943 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.804225', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBApkJcznsj8WnsOIJ8SVPpV7JNCtjdQODYCXrZAnLG84xAi1O0ghj/GK+p0bBV5RAYmlVrklAmLd1GB1U//t3RA='} 2024-04-25T14:40:13.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.807798', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAZA+gtfIGwyGI91oCGb3Byh26KYW3iDJJL9Jmgauyojp5NdDXjBcHsVdIM1RTSBz3p11BZnESuxL4IVXfU8FA='} 2024-04-25T14:40:13.965 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T14:40:13.972 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T14:40:13.972 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T14:40:13.972 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T14:40:13.972 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T14:40:14.068 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff9e0350d0>, signals=[15]) 2024-04-25T14:40:14.068 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T14:40:14.075 INFO:teuthology.task.internal:Opening connections... 2024-04-25T14:40:14.075 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T14:40:14.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:40:14.153 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T14:40:14.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:40:14.226 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-04-25T14:40:14.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:40:14.301 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T14:40:14.308 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-25T14:40:14.326 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-25T14:40:14.327 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-25T14:40:14.384 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-25T14:40:14.384 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-25T14:40:14.384 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-25T14:40:14.384 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:40:14.385 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:40:14.386 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-25T14:40:14.410 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-25T14:40:14.427 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-25T14:40:14.427 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-25T14:40:14.485 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:40:14.486 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:40:14.486 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:40:14.486 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:40:14.486 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:40:14.486 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:40:14.486 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-25T14:40:14.509 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-04-25T14:40:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-04-25T14:40:14.527 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-04-25T14:40:14.586 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:40:14.587 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:40:14.588 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-04-25T14:40:14.609 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T14:40:14.619 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T14:40:14.625 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T14:40:14.625 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:40:14.627 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:40:14.629 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:40:14.645 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T14:40:14.651 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T14:40:14.651 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:40:14.685 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:40:14.688 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:40:14.740 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T14:40:14.816 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T14:40:14.829 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T14:40:14.829 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T14:40:14.829 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T14:40:14.830 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:40:14.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:40:14.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T14:40:15.098 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:40:15.098 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T14:40:15.098 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T14:40:15.116 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-25T14:40:15.116 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:40:15.116 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T14:40:15.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum upgrade -y linux-firmware 2024-04-25T14:40:17.595 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-25T14:40:19.613 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T14:40:23.822 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T14:40:25.463 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T14:40:26.614 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T14:40:31.764 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T14:40:33.857 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T14:40:33.858 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-25T14:40:33.858 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T14:40:33.969 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:40:33.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:40:34.201 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T14:40:34.201 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T14:40:34.203 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi019.front.sepia.ceph.com to complete... 2024-04-25T14:40:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25 14:40:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T14:40:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:FINISHED --2024-04-25 14:40:46-- 2024-04-25T14:40:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:Total wall clock time: 12s 2024-04-25T14:40:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:Downloaded: 1 files, 341M in 12s (28.4 MB/s) 2024-04-25T14:40:46.584 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:40:46.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T14:40:48.768 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying... ######################################## 2024-04-25T14:40:48.921 INFO:teuthology.orchestra.run.smithi019.stdout:Preparing... ######################################## 2024-04-25T14:40:49.027 INFO:teuthology.orchestra.run.smithi019.stdout:Updating / installing... 2024-04-25T14:40:57.334 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T14:41:37.558 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T14:41:37.559 DEBUG:teuthology.orchestra.run.smithi019:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T14:41:37.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T14:41:37.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T14:41:37.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T14:41:37.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T14:41:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T14:41:37.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T14:41:37.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T14:41:37.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T14:41:37.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T14:41:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T14:41:37.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T14:41:37.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T14:41:37.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T14:41:37.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T14:41:37.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T14:41:37.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T14:41:37.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T14:41:37.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T14:41:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T14:41:37.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T14:41:37.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T14:41:37.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T14:41:37.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T14:41:37.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T14:41:37.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T14:41:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T14:41:37.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T14:41:37.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T14:41:37.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T14:41:37.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T14:41:37.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T14:41:37.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T14:41:37.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T14:41:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T14:41:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T14:41:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T14:41:37.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T14:41:37.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T14:41:37.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T14:41:37.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T14:41:37.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T14:41:37.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T14:41:37.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T14:41:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T14:41:37.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T14:41:37.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T14:41:37.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T14:41:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T14:41:37.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T14:41:37.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T14:41:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T14:41:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T14:41:37.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T14:41:37.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T14:41:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T14:41:37.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T14:41:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T14:41:37.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T14:41:37.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T14:41:37.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T14:41:37.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T14:41:37.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T14:41:37.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T14:41:37.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T14:41:37.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T14:41:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T14:41:37.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T14:41:37.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T14:41:37.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T14:41:37.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T14:41:37.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T14:41:37.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T14:41:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T14:41:37.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T14:41:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T14:41:37.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T14:41:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T14:41:37.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T14:41:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T14:41:37.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T14:41:37.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T14:41:37.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T14:41:37.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T14:41:37.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T14:41:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T14:41:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T14:41:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T14:41:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T14:41:37.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T14:41:37.733 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T14:41:37.734 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T14:41:37.735 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T14:41:37.736 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T14:41:37.737 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T14:41:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T14:41:37.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T14:41:37.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T14:41:37.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T14:41:37.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T14:41:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T14:41:37.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T14:41:37.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T14:41:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T14:41:37.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T14:41:37.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T14:41:37.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T14:41:37.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T14:41:37.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T14:41:37.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T14:41:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T14:41:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T14:41:37.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T14:41:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T14:41:37.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T14:41:37.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T14:41:37.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T14:41:37.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T14:41:37.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T14:41:37.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T14:41:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T14:41:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T14:41:37.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T14:41:37.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T14:41:37.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T14:41:37.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T14:41:37.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T14:41:37.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T14:41:37.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T14:41:37.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T14:41:37.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T14:41:37.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T14:41:37.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T14:41:37.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T14:41:37.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T14:41:37.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T14:41:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T14:41:37.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T14:41:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T14:41:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T14:41:37.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T14:41:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T14:41:37.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T14:41:37.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T14:41:37.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T14:41:37.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T14:41:37.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T14:41:37.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T14:41:37.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T14:41:37.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:41:37.796 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.796 DEBUG:teuthology.orchestra.run.smithi019:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T14:41:37.819 INFO:teuthology.orchestra.run.smithi019.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout:for file in vmlinuz System.map config; do 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: fi 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:41:37.820 INFO:teuthology.orchestra.run.smithi019.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.821 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T14:41:37.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-04-25T14:41:37.903 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-04-25T14:41:37.903 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-04-25T14:41:37.903 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-04-25T14:41:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:41:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T14:41:37.906 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T14:41:37.906 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:37.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T14:41:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-04-25T14:41:39.958 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T14:41:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-04-25T14:41:40.002 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-04-25T14:41:40.017 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.Y8qkgXBPtF 2024-04-25T14:41:40.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y8qkgXBPtF 2024-04-25T14:41:40.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.Y8qkgXBPtF 2024-04-25T14:41:40.225 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.Y8qkgXBPtF is 6KB 2024-04-25T14:41:40.274 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.Y8qkgXBPtF 2024-04-25T14:41:40.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T14:41:40.358 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T14:41:40.359 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T14:41:40.359 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T14:41:40.359 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T14:41:40.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:40.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-04-25T14:42:10.494 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T14:42:10.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:42:10.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:42:29.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-04-25T14:42:38.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:42:38.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:43:38.027 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T14:43:50.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:43:50.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:43:50.462 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T14:43:51.164 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:43:51.164 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T14:43:51.164 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T14:43:51.215 INFO:teuthology.orchestra.run.smithi019.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T14:43:51.215 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T14:43:51.215 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T14:43:51.215 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T14:43:52.216 DEBUG:teuthology.parallel:result is None 2024-04-25T14:43:52.217 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T14:43:52.225 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T14:43:52.225 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:43:52.233 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:43:52.236 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:43:52.285 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T14:43:52.292 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T14:43:52.299 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T14:43:52.299 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:43:52.305 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:43:52.307 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:43:52.378 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T14:43:52.385 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T14:43:52.385 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:43:52.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:43:52.402 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:43:52.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:43:52.419 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:43:52.427 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:43:52.446 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:43:52.493 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.496 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.510 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.518 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.571 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.624 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:43:52.629 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T14:43:52.636 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T14:43:52.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:43:52.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:43:52.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:43:52.729 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T14:43:52.739 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T14:43:52.739 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:43:52.745 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:43:52.748 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:43:52.794 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:52.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:52.979 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.156 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:43:53.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:43:53.241 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:53.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:53.345 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.470 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:43:53.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:43:53.536 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:53.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:43:53.855 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.896 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:43:53.943 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:43:53.943 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:43:54.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-25T14:43:54.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-25T14:43:54.019 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-04-25T14:43:54.065 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:43:54.089 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:43:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:43:54.573 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T14:43:54.581 INFO:teuthology.task.internal:Starting timer... 2024-04-25T14:43:54.582 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T14:43:54.592 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T14:43:54.600 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T14:43:54.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-25T14:43:54.707 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-25T14:43:54.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-25T14:43:54.779 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-25T14:43:54.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-04-25T14:43:54.859 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-04-25T14:43:54.861 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T14:43:54.861 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-25T14:43:54.918 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-25T14:43:54.918 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-25T14:43:54.945 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-25T14:43:54.946 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-04-25T14:43:54.967 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-04-25T14:43:54.968 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-04-25T14:43:54.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:43:55.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:43:55.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:43:55.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:43:55.078 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:43:55.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:43:55.104 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T14:43:55.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:43:55.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:43:55.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:43:55.235 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T14:43:55.245 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T14:43:55.251 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T14:43:55.374 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T14:43:55.389 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T14:43:55.391 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi083.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T14:49:07.486 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-04-25T14:49:07.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:49:07.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:49:07.601 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T14:49:07.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T14:49:07.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T14:49:07.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:49:07.834 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-25T14:49:07.909 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T14:49:07.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-25T14:49:07.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:49:07.984 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-25T14:49:08.066 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-25T14:49:08.066 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T14:49:08.076 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T14:49:08.076 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:49:08.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:49:08.084 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:49:08.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:49:08.086 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:49:08.087 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:49:08.114 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:49:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:49:08.135 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:49:08.149 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:49:08.152 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-25T14:49:08.163 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-25T14:49:08.173 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-04-25T14:49:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:49:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:49:08.183 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-04-25T14:49:08.190 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:49:08.197 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:49:08.210 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:49:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:49:08.242 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.274 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:49:08.278 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:49:08.278 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-25T14:49:08.278 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.279 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.279 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.279 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.297 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-25T14:49:08.332 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-25T14:49:08.376 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:49:08.420 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:49:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:49:08.588 INFO:teuthology.run_tasks:Running task install... 2024-04-25T14:49:08.597 DEBUG:teuthology.task.install:project ceph 2024-04-25T14:49:08.597 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T14:49:08.597 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T14:49:08.597 INFO:teuthology.task.install:Using flavor: default 2024-04-25T14:49:08.602 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T14:49:08.602 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T14:49:08.603 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:49:08.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:49:08.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:49:08.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:49:08.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:49:08.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:49:08.821 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:49:08.821 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:49:08.839 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:49:08.839 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:49:08.842 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:49:08.843 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:49:08.935 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:49:08.936 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:49:08.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:49:08.951 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:49:08.951 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:49:08.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:49:08.956 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:49:08.956 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:49:08.956 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:49:08.979 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:49:08.979 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:49:08.982 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:49:08.982 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:49:09.031 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:49:09.031 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:49:09.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:49:09.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:49:09.121 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:49:09.122 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:49:09.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:49:09.185 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-04-25T14:49:09.186 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-25T14:49:09.187 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-04-25T14:49:09.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-25T14:49:09.288 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:49:09.372 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-25T14:49:09.377 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-25T14:49:09.410 INFO:teuthology.orchestra.run.smithi102.stdout:48 files removed 2024-04-25T14:49:09.414 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-25T14:49:09.438 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:49:09.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:49:09.713 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-25T14:49:09.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:49:10.067 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 225 kB/s | 83 kB 00:00 2024-04-25T14:49:10.165 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 180 kB/s | 83 kB 00:00 2024-04-25T14:49:10.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-25T14:49:10.524 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T14:49:10.597 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 213 kB/s | 83 kB 00:00 2024-04-25T14:49:10.759 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-04-25T14:49:10.949 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T14:49:11.098 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T14:49:11.562 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T14:49:11.682 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-25T14:49:11.977 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-25T14:49:12.551 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T14:49:13.601 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-25T14:49:13.979 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T14:49:14.921 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T14:49:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-25T14:49:18.088 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T14:49:19.246 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T14:49:19.613 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-25T14:49:20.078 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-25T14:49:21.716 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-25T14:49:23.339 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 21 MB 00:03 2024-04-25T14:49:23.779 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-04-25T14:49:24.117 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-04-25T14:49:28.355 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-25T14:49:28.846 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T14:49:30.343 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T14:49:30.474 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:30.475 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:30.485 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:49:30.486 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:49:30.487 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:49:30.488 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:49:30.489 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:49:30.492 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:49:30.492 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:49:30.552 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-25T14:49:30.565 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:49:30.566 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:49:30.567 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:49:30.568 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:49:30.569 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:49:30.570 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:30.571 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:49:30.572 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:49:30.573 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:49:30.574 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:49:30.575 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:49:30.576 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:49:30.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 4 Packages 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 222 M 2024-04-25T14:49:30.578 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-25T14:49:30.990 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:30.991 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:31.001 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:49:31.002 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:49:31.003 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:49:31.004 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:49:31.005 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:49:31.007 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:49:31.008 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:49:31.069 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-25T14:49:31.088 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T14:49:31.088 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-25T14:49:31.088 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T14:49:31.088 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-25T14:49:31.088 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:49:31.089 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:49:31.090 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-25T14:49:31.091 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:49:31.092 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:49:31.093 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:49:31.094 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:49:31.095 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:49:31.096 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:49:31.097 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:49:31.098 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:49:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:49:31.100 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:49:31.101 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 4 Packages 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 222 M 2024-04-25T14:49:31.102 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-25T14:49:31.432 INFO:teuthology.orchestra.run.smithi102.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-25T14:49:31.649 INFO:teuthology.orchestra.run.smithi083.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-25T14:49:31.890 INFO:teuthology.orchestra.run.smithi102.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 890 kB 00:00 2024-04-25T14:49:31.933 INFO:teuthology.orchestra.run.smithi083.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 890 kB 00:00 2024-04-25T14:49:31.990 INFO:teuthology.orchestra.run.smithi102.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-25T14:49:31.999 INFO:teuthology.orchestra.run.smithi083.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T14:49:32.149 INFO:teuthology.orchestra.run.smithi102.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.6 MB/s | 5.4 MB 00:00 2024-04-25T14:49:32.366 INFO:teuthology.orchestra.run.smithi102.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-25T14:49:32.466 INFO:teuthology.orchestra.run.smithi083.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 2.4 MB 00:00 2024-04-25T14:49:32.625 INFO:teuthology.orchestra.run.smithi083.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.6 MB/s | 5.4 MB 00:01 2024-04-25T14:49:32.775 INFO:teuthology.orchestra.run.smithi102.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.0 MB/s | 2.4 MB 00:00 2024-04-25T14:49:32.825 INFO:teuthology.orchestra.run.smithi083.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.2 MB/s | 1.5 MB 00:00 2024-04-25T14:49:32.883 INFO:teuthology.orchestra.run.smithi102.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 9.1 MB/s | 4.7 MB 00:00 2024-04-25T14:49:32.887 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:32.888 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:49:32.909 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:49:32.911 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:49:32.912 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:49:32.913 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:49:32.916 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:49:32.919 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:49:32.920 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:49:32.995 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:49:33.020 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:49:33.021 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:49:33.022 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:49:33.023 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:49:33.024 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:49:33.025 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:49:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:49:33.027 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:49:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:49:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:49:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:49:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:49:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T14:49:33.033 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-25T14:49:33.034 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-25T14:49:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:49:33.034 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 222 M 2024-04-25T14:49:33.034 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T14:49:33.401 INFO:teuthology.orchestra.run.smithi083.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.1 MB/s | 4.7 MB 00:00 2024-04-25T14:49:34.003 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-25T14:49:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 7.4 MB/s | 11 MB 00:01 2024-04-25T14:49:34.400 INFO:teuthology.orchestra.run.smithi102.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 275 kB/s | 25 kB 00:00 2024-04-25T14:49:35.144 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 780 kB/s | 890 kB 00:01 2024-04-25T14:49:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 5.9 MB/s | 11 MB 00:01 2024-04-25T14:49:35.293 INFO:teuthology.orchestra.run.smithi083.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 275 kB/s | 25 kB 00:00 2024-04-25T14:49:35.328 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 795 kB/s | 145 kB 00:00 2024-04-25T14:49:36.278 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-25T14:49:36.443 INFO:teuthology.orchestra.run.smithi083.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.7 MB/s | 17 MB 00:03 2024-04-25T14:49:36.552 INFO:teuthology.orchestra.run.smithi083.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 225 kB 00:00 2024-04-25T14:49:36.644 INFO:teuthology.orchestra.run.smithi083.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 294 kB/s | 27 kB 00:00 2024-04-25T14:49:36.695 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 1.5 MB 00:00 2024-04-25T14:49:36.710 INFO:teuthology.orchestra.run.smithi083.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 478 kB/s | 32 kB 00:00 2024-04-25T14:49:36.928 INFO:teuthology.orchestra.run.smithi083.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.4 MB/s | 747 kB 00:00 2024-04-25T14:49:37.003 INFO:teuthology.orchestra.run.smithi083.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-25T14:49:37.078 INFO:teuthology.orchestra.run.smithi083.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.6 MB/s | 126 kB 00:00 2024-04-25T14:49:37.203 INFO:teuthology.orchestra.run.smithi083.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.9 MB/s | 500 kB 00:00 2024-04-25T14:49:37.242 INFO:teuthology.orchestra.run.smithi102.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 21 MB 00:06 2024-04-25T14:49:37.426 INFO:teuthology.orchestra.run.smithi102.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 225 kB 00:00 2024-04-25T14:49:37.534 INFO:teuthology.orchestra.run.smithi102.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 250 kB/s | 27 kB 00:00 2024-04-25T14:49:37.554 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-25T14:49:37.635 INFO:teuthology.orchestra.run.smithi102.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 317 kB/s | 32 kB 00:00 2024-04-25T14:49:37.926 INFO:teuthology.orchestra.run.smithi102.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 747 kB 00:00 2024-04-25T14:49:38.035 INFO:teuthology.orchestra.run.smithi102.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T14:49:38.161 INFO:teuthology.orchestra.run.smithi102.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.0 MB/s | 126 kB 00:00 2024-04-25T14:49:38.203 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-25T14:49:38.245 INFO:teuthology.orchestra.run.smithi083.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 5.4 MB 00:01 2024-04-25T14:49:38.311 INFO:teuthology.orchestra.run.smithi102.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.3 MB/s | 500 kB 00:00 2024-04-25T14:49:38.337 INFO:teuthology.orchestra.run.smithi083.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 493 kB/s | 45 kB 00:00 2024-04-25T14:49:38.430 INFO:teuthology.orchestra.run.smithi083.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T14:49:38.521 INFO:teuthology.orchestra.run.smithi083.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T14:49:38.621 INFO:teuthology.orchestra.run.smithi083.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.1 MB/s | 320 kB 00:00 2024-04-25T14:49:38.705 INFO:teuthology.orchestra.run.smithi083.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-25T14:49:38.764 INFO:teuthology.orchestra.run.smithi083.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-25T14:49:38.839 INFO:teuthology.orchestra.run.smithi083.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T14:49:38.870 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 21 MB 00:05 2024-04-25T14:49:38.970 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 255 kB/s | 25 kB 00:00 2024-04-25T14:49:39.420 INFO:teuthology.orchestra.run.smithi102.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 17 MB 00:06 2024-04-25T14:49:39.489 INFO:teuthology.orchestra.run.smithi083.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.8 MB/s | 3.1 MB 00:00 2024-04-25T14:49:39.520 INFO:teuthology.orchestra.run.smithi102.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 452 kB/s | 45 kB 00:00 2024-04-25T14:49:39.590 INFO:teuthology.orchestra.run.smithi083.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T14:49:39.620 INFO:teuthology.orchestra.run.smithi102.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T14:49:39.665 INFO:teuthology.orchestra.run.smithi083.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 456 kB/s | 34 kB 00:00 2024-04-25T14:49:39.729 INFO:teuthology.orchestra.run.smithi102.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T14:49:39.749 INFO:teuthology.orchestra.run.smithi083.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-25T14:49:39.871 INFO:teuthology.orchestra.run.smithi102.stdout:(22/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-25T14:49:39.946 INFO:teuthology.orchestra.run.smithi102.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 320 kB 00:00 2024-04-25T14:49:39.996 INFO:teuthology.orchestra.run.smithi102.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 298 kB 00:00 2024-04-25T14:49:40.055 INFO:teuthology.orchestra.run.smithi102.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 917 kB/s | 99 kB 00:00 2024-04-25T14:49:40.105 INFO:teuthology.orchestra.run.smithi102.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 786 kB/s | 85 kB 00:00 2024-04-25T14:49:40.222 INFO:teuthology.orchestra.run.smithi102.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 170 kB 00:00 2024-04-25T14:49:40.331 INFO:teuthology.orchestra.run.smithi102.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 316 kB/s | 34 kB 00:00 2024-04-25T14:49:40.448 INFO:teuthology.orchestra.run.smithi102.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.2 MB/s | 142 kB 00:00 2024-04-25T14:49:40.507 INFO:teuthology.orchestra.run.smithi083.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.7 MB/s | 3.5 MB 00:00 2024-04-25T14:49:40.696 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.3 MB/s | 11 MB 00:02 2024-04-25T14:49:40.796 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T14:49:40.913 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 233 kB/s | 27 kB 00:00 2024-04-25T14:49:41.050 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 233 kB/s | 32 kB 00:00 2024-04-25T14:49:41.395 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 747 kB 00:00 2024-04-25T14:49:41.481 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-25T14:49:41.523 INFO:teuthology.orchestra.run.smithi102.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.3 MB/s | 3.5 MB 00:01 2024-04-25T14:49:41.531 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-25T14:49:41.582 INFO:teuthology.orchestra.run.smithi083.stdout:(31/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.1 MB/s | 21 MB 00:10 2024-04-25T14:49:41.598 INFO:teuthology.orchestra.run.smithi102.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.0 MB/s | 3.1 MB 00:01 2024-04-25T14:49:41.647 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-25T14:49:41.790 INFO:teuthology.orchestra.run.smithi102.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.3 MB/s | 249 kB 00:00 2024-04-25T14:49:41.808 INFO:teuthology.orchestra.run.smithi083.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.1 MB/s | 249 kB 00:00 2024-04-25T14:49:41.882 INFO:teuthology.orchestra.run.smithi102.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 539 kB/s | 49 kB 00:00 2024-04-25T14:49:41.908 INFO:teuthology.orchestra.run.smithi083.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 494 kB/s | 49 kB 00:00 2024-04-25T14:49:41.983 INFO:teuthology.orchestra.run.smithi102.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 164 kB/s | 16 kB 00:00 2024-04-25T14:49:42.067 INFO:teuthology.orchestra.run.smithi083.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-25T14:49:42.092 INFO:teuthology.orchestra.run.smithi083.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 90 kB/s | 16 kB 00:00 2024-04-25T14:49:42.150 INFO:teuthology.orchestra.run.smithi102.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 273 kB 00:00 2024-04-25T14:49:42.150 INFO:teuthology.orchestra.run.smithi083.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-25T14:49:42.234 INFO:teuthology.orchestra.run.smithi083.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 219 kB/s | 18 kB 00:00 2024-04-25T14:49:42.343 INFO:teuthology.orchestra.run.smithi083.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T14:49:42.426 INFO:teuthology.orchestra.run.smithi083.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T14:49:42.468 INFO:teuthology.orchestra.run.smithi083.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T14:49:42.494 INFO:teuthology.orchestra.run.smithi083.stdout:(41/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.9 MB/s | 762 kB 00:00 2024-04-25T14:49:42.519 INFO:teuthology.orchestra.run.smithi083.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T14:49:42.561 INFO:teuthology.orchestra.run.smithi083.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 19 MB/s | 1.2 MB 00:00 2024-04-25T14:49:42.575 INFO:teuthology.orchestra.run.smithi102.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.8 MB/s | 762 kB 00:00 2024-04-25T14:49:42.586 INFO:teuthology.orchestra.run.smithi083.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T14:49:42.612 INFO:teuthology.orchestra.run.smithi083.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T14:49:42.637 INFO:teuthology.orchestra.run.smithi083.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T14:49:42.662 INFO:teuthology.orchestra.run.smithi083.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T14:49:42.667 INFO:teuthology.orchestra.run.smithi102.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T14:49:42.687 INFO:teuthology.orchestra.run.smithi083.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T14:49:42.721 INFO:teuthology.orchestra.run.smithi083.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T14:49:42.746 INFO:teuthology.orchestra.run.smithi083.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T14:49:42.771 INFO:teuthology.orchestra.run.smithi083.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-25T14:49:42.797 INFO:teuthology.orchestra.run.smithi083.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T14:49:42.822 INFO:teuthology.orchestra.run.smithi083.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T14:49:42.825 INFO:teuthology.orchestra.run.smithi102.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 508 kB/s | 80 kB 00:00 2024-04-25T14:49:42.881 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 4.3 MB/s | 5.4 MB 00:01 2024-04-25T14:49:42.906 INFO:teuthology.orchestra.run.smithi083.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T14:49:42.923 INFO:teuthology.orchestra.run.smithi083.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 185 kB/s | 18 kB 00:00 2024-04-25T14:49:42.940 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 787 kB/s | 45 kB 00:00 2024-04-25T14:49:42.940 INFO:teuthology.orchestra.run.smithi083.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-25T14:49:42.974 INFO:teuthology.orchestra.run.smithi083.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T14:49:42.984 INFO:teuthology.orchestra.run.smithi102.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-04-25T14:49:42.990 INFO:teuthology.orchestra.run.smithi083.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-04-25T14:49:43.015 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-25T14:49:43.033 INFO:teuthology.orchestra.run.smithi083.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T14:49:43.034 INFO:teuthology.orchestra.run.smithi102.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T14:49:43.058 INFO:teuthology.orchestra.run.smithi083.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T14:49:43.075 INFO:teuthology.orchestra.run.smithi083.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 623 kB/s | 25 kB 00:00 2024-04-25T14:49:43.082 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T14:49:43.085 INFO:teuthology.orchestra.run.smithi102.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T14:49:43.092 INFO:teuthology.orchestra.run.smithi083.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T14:49:43.125 INFO:teuthology.orchestra.run.smithi083.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T14:49:43.150 INFO:teuthology.orchestra.run.smithi083.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T14:49:43.160 INFO:teuthology.orchestra.run.smithi102.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 706 kB/s | 35 kB 00:00 2024-04-25T14:49:43.190 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-25T14:49:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T14:49:43.277 INFO:teuthology.orchestra.run.smithi102.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T14:49:43.299 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 298 kB 00:00 2024-04-25T14:49:43.309 INFO:teuthology.orchestra.run.smithi083.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-25T14:49:43.336 INFO:teuthology.orchestra.run.smithi102.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T14:49:43.366 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.5 MB/s | 99 kB 00:00 2024-04-25T14:49:43.411 INFO:teuthology.orchestra.run.smithi102.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T14:49:43.475 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 793 kB/s | 85 kB 00:00 2024-04-25T14:49:43.478 INFO:teuthology.orchestra.run.smithi102.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-25T14:49:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.8 MB/s | 1.6 MB 00:00 2024-04-25T14:49:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T14:49:43.537 INFO:teuthology.orchestra.run.smithi102.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T14:49:43.560 INFO:teuthology.orchestra.run.smithi083.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T14:49:43.570 INFO:teuthology.orchestra.run.smithi102.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T14:49:43.596 INFO:teuthology.orchestra.run.smithi102.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 863 kB/s | 21 kB 00:00 2024-04-25T14:49:43.638 INFO:teuthology.orchestra.run.smithi102.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T14:49:43.663 INFO:teuthology.orchestra.run.smithi102.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 33 kB 00:00 2024-04-25T14:49:43.703 INFO:teuthology.orchestra.run.smithi083.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-04-25T14:49:43.736 INFO:teuthology.orchestra.run.smithi083.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-25T14:49:43.755 INFO:teuthology.orchestra.run.smithi102.stdout:(53/152): ceph-mgr-diskprediction-local-19.0.0- 3.3 MB/s | 7.4 MB 00:02 2024-04-25T14:49:43.762 INFO:teuthology.orchestra.run.smithi083.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T14:49:43.895 INFO:teuthology.orchestra.run.smithi083.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-25T14:49:43.914 INFO:teuthology.orchestra.run.smithi102.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-25T14:49:43.920 INFO:teuthology.orchestra.run.smithi083.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T14:49:43.939 INFO:teuthology.orchestra.run.smithi102.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 101 kB/s | 18 kB 00:00 2024-04-25T14:49:43.964 INFO:teuthology.orchestra.run.smithi102.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T14:49:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T14:49:43.996 INFO:teuthology.orchestra.run.smithi083.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T14:49:44.000 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 5.9 MB/s | 3.1 MB 00:00 2024-04-25T14:49:44.015 INFO:teuthology.orchestra.run.smithi102.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T14:49:44.040 INFO:teuthology.orchestra.run.smithi102.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T14:49:44.046 INFO:teuthology.orchestra.run.smithi083.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-04-25T14:49:44.067 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T14:49:44.071 INFO:teuthology.orchestra.run.smithi083.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-25T14:49:44.074 INFO:teuthology.orchestra.run.smithi102.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T14:49:44.089 INFO:teuthology.orchestra.run.smithi083.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T14:49:44.099 INFO:teuthology.orchestra.run.smithi102.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-04-25T14:49:44.110 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 832 kB/s | 34 kB 00:00 2024-04-25T14:49:44.122 INFO:teuthology.orchestra.run.smithi083.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T14:49:44.124 INFO:teuthology.orchestra.run.smithi102.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T14:49:44.150 INFO:teuthology.orchestra.run.smithi102.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T14:49:44.175 INFO:teuthology.orchestra.run.smithi102.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T14:49:44.200 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T14:49:44.208 INFO:teuthology.orchestra.run.smithi102.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-04-25T14:49:44.256 INFO:teuthology.orchestra.run.smithi083.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-04-25T14:49:44.281 INFO:teuthology.orchestra.run.smithi083.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T14:49:44.315 INFO:teuthology.orchestra.run.smithi083.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T14:49:44.340 INFO:teuthology.orchestra.run.smithi083.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T14:49:44.365 INFO:teuthology.orchestra.run.smithi083.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T14:49:44.391 INFO:teuthology.orchestra.run.smithi083.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T14:49:44.425 INFO:teuthology.orchestra.run.smithi083.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T14:49:44.449 INFO:teuthology.orchestra.run.smithi083.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T14:49:44.467 INFO:teuthology.orchestra.run.smithi083.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T14:49:44.483 INFO:teuthology.orchestra.run.smithi083.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T14:49:44.643 INFO:teuthology.orchestra.run.smithi083.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 264 kB/s | 42 kB 00:00 2024-04-25T14:49:44.644 INFO:teuthology.orchestra.run.smithi102.stdout:(66/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.8 MB/s | 49 MB 00:10 2024-04-25T14:49:44.684 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.3 MB/s | 3.5 MB 00:00 2024-04-25T14:49:44.853 INFO:teuthology.orchestra.run.smithi102.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-04-25T14:49:44.929 INFO:teuthology.orchestra.run.smithi102.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.3 MB/s | 4.7 MB 00:00 2024-04-25T14:49:44.970 INFO:teuthology.orchestra.run.smithi102.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-04-25T14:49:45.003 INFO:teuthology.orchestra.run.smithi102.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.1 MB/s | 911 kB 00:00 2024-04-25T14:49:45.028 INFO:teuthology.orchestra.run.smithi102.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T14:49:45.093 INFO:teuthology.orchestra.run.smithi083.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-25T14:49:45.110 INFO:teuthology.orchestra.run.smithi083.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 95 kB/s | 44 kB 00:00 2024-04-25T14:49:45.129 INFO:teuthology.orchestra.run.smithi102.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T14:49:45.154 INFO:teuthology.orchestra.run.smithi102.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 698 kB/s | 87 kB 00:00 2024-04-25T14:49:45.255 INFO:teuthology.orchestra.run.smithi102.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-25T14:49:45.288 INFO:teuthology.orchestra.run.smithi102.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T14:49:45.405 INFO:teuthology.orchestra.run.smithi102.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-25T14:49:45.430 INFO:teuthology.orchestra.run.smithi102.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-04-25T14:49:45.447 INFO:teuthology.orchestra.run.smithi102.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 26 kB 00:00 2024-04-25T14:49:45.464 INFO:teuthology.orchestra.run.smithi102.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 330 kB/s | 19 kB 00:00 2024-04-25T14:49:45.468 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.4 MB/s | 7.4 MB 00:00 2024-04-25T14:49:45.481 INFO:teuthology.orchestra.run.smithi102.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-25T14:49:45.506 INFO:teuthology.orchestra.run.smithi102.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T14:49:45.532 INFO:teuthology.orchestra.run.smithi102.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-25T14:49:45.543 INFO:teuthology.orchestra.run.smithi083.stdout:(94/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.8 MB/s | 49 MB 00:10 2024-04-25T14:49:45.549 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.1 MB/s | 249 kB 00:00 2024-04-25T14:49:45.568 INFO:teuthology.orchestra.run.smithi083.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 464 kB/s | 221 kB 00:00 2024-04-25T14:49:45.574 INFO:teuthology.orchestra.run.smithi102.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 222 kB 00:00 2024-04-25T14:49:45.594 INFO:teuthology.orchestra.run.smithi083.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 74 kB/s | 36 kB 00:00 2024-04-25T14:49:45.602 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 953 kB/s | 49 kB 00:00 2024-04-25T14:49:45.619 INFO:teuthology.orchestra.run.smithi083.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 634 kB/s | 47 kB 00:00 2024-04-25T14:49:45.624 INFO:teuthology.orchestra.run.smithi102.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 867 kB/s | 77 kB 00:00 2024-04-25T14:49:45.652 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 334 kB/s | 16 kB 00:00 2024-04-25T14:49:45.653 INFO:teuthology.orchestra.run.smithi083.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T14:49:45.678 INFO:teuthology.orchestra.run.smithi083.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 691 kB/s | 58 kB 00:00 2024-04-25T14:49:45.691 INFO:teuthology.orchestra.run.smithi102.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T14:49:45.703 INFO:teuthology.orchestra.run.smithi083.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-25T14:49:45.716 INFO:teuthology.orchestra.run.smithi102.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 302 kB/s | 42 kB 00:00 2024-04-25T14:49:45.719 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.1 MB/s | 273 kB 00:00 2024-04-25T14:49:45.728 INFO:teuthology.orchestra.run.smithi083.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 849 kB/s | 64 kB 00:00 2024-04-25T14:49:45.750 INFO:teuthology.orchestra.run.smithi102.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-25T14:49:45.754 INFO:teuthology.orchestra.run.smithi083.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-04-25T14:49:45.775 INFO:teuthology.orchestra.run.smithi102.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T14:49:45.779 INFO:teuthology.orchestra.run.smithi083.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T14:49:45.808 INFO:teuthology.orchestra.run.smithi102.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-25T14:49:45.821 INFO:teuthology.orchestra.run.smithi083.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T14:49:45.852 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T14:49:45.858 INFO:teuthology.orchestra.run.smithi102.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-04-25T14:49:45.884 INFO:teuthology.orchestra.run.smithi102.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-04-25T14:49:45.909 INFO:teuthology.orchestra.run.smithi102.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-25T14:49:45.919 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 278 kB/s | 18 kB 00:00 2024-04-25T14:49:45.942 INFO:teuthology.orchestra.run.smithi102.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-04-25T14:49:45.985 INFO:teuthology.orchestra.run.smithi102.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.0 MB/s | 221 kB 00:00 2024-04-25T14:49:46.010 INFO:teuthology.orchestra.run.smithi102.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 561 kB/s | 36 kB 00:00 2024-04-25T14:49:46.069 INFO:teuthology.orchestra.run.smithi102.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 585 kB/s | 47 kB 00:00 2024-04-25T14:49:46.079 INFO:teuthology.orchestra.run.smithi083.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T14:49:46.119 INFO:teuthology.orchestra.run.smithi102.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-25T14:49:46.120 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 404 kB/s | 80 kB 00:00 2024-04-25T14:49:46.203 INFO:teuthology.orchestra.run.smithi102.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 439 kB/s | 58 kB 00:00 2024-04-25T14:49:46.361 INFO:teuthology.orchestra.run.smithi102.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-25T14:49:46.387 INFO:teuthology.orchestra.run.smithi102.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 354 kB/s | 94 kB 00:00 2024-04-25T14:49:46.412 INFO:teuthology.orchestra.run.smithi102.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 308 kB/s | 64 kB 00:00 2024-04-25T14:49:46.437 INFO:teuthology.orchestra.run.smithi102.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T14:49:46.438 INFO:teuthology.orchestra.run.smithi083.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 470 kB/s | 309 kB 00:00 2024-04-25T14:49:46.462 INFO:teuthology.orchestra.run.smithi102.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-04-25T14:49:46.496 INFO:teuthology.orchestra.run.smithi102.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T14:49:46.497 INFO:teuthology.orchestra.run.smithi083.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 766 kB/s | 548 kB 00:00 2024-04-25T14:49:46.547 INFO:teuthology.orchestra.run.smithi083.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T14:49:46.597 INFO:teuthology.orchestra.run.smithi083.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 483 kB/s | 49 kB 00:00 2024-04-25T14:49:46.656 INFO:teuthology.orchestra.run.smithi083.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T14:49:46.714 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.3 MB/s | 49 MB 00:07 2024-04-25T14:49:46.714 INFO:teuthology.orchestra.run.smithi083.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T14:49:46.754 INFO:teuthology.orchestra.run.smithi102.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T14:49:46.789 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-25T14:49:46.822 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-25T14:49:46.857 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-04-25T14:49:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-04-25T14:49:46.999 INFO:teuthology.orchestra.run.smithi083.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-04-25T14:49:47.024 INFO:teuthology.orchestra.run.smithi083.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-25T14:49:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-25T14:49:47.071 INFO:teuthology.orchestra.run.smithi102.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 509 kB/s | 309 kB 00:00 2024-04-25T14:49:47.082 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.8 MB/s | 17 MB 00:09 2024-04-25T14:49:47.091 INFO:teuthology.orchestra.run.smithi083.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-25T14:49:47.116 INFO:teuthology.orchestra.run.smithi083.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T14:49:47.138 INFO:teuthology.orchestra.run.smithi102.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 813 kB/s | 548 kB 00:00 2024-04-25T14:49:47.140 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-25T14:49:47.165 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 348 kB/s | 106 kB 00:00 2024-04-25T14:49:47.172 INFO:teuthology.orchestra.run.smithi102.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T14:49:47.199 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-25T14:49:47.208 INFO:teuthology.orchestra.run.smithi083.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T14:49:47.232 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-25T14:49:47.250 INFO:teuthology.orchestra.run.smithi083.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T14:49:47.255 INFO:teuthology.orchestra.run.smithi102.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-04-25T14:49:47.266 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 215 kB 00:00 2024-04-25T14:49:47.275 INFO:teuthology.orchestra.run.smithi083.stdout:(120/152): python3-cheroot-8.6.0-4.el9.noarch.r 759 kB/s | 172 kB 00:00 2024-04-25T14:49:47.281 INFO:teuthology.orchestra.run.smithi102.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T14:49:47.301 INFO:teuthology.orchestra.run.smithi083.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-04-25T14:49:47.324 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-25T14:49:47.350 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-25T14:49:47.372 INFO:teuthology.orchestra.run.smithi102.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T14:49:47.376 INFO:teuthology.orchestra.run.smithi083.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-25T14:49:47.408 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 6.8 MB/s | 943 kB 00:00 2024-04-25T14:49:47.409 INFO:teuthology.orchestra.run.smithi083.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T14:49:47.425 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 218 kB/s | 21 kB 00:00 2024-04-25T14:49:47.459 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-25T14:49:47.476 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 709 kB/s | 33 kB 00:00 2024-04-25T14:49:47.485 INFO:teuthology.orchestra.run.smithi083.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-25T14:49:47.527 INFO:teuthology.orchestra.run.smithi083.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T14:49:47.559 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-04-25T14:49:47.565 INFO:teuthology.orchestra.run.smithi102.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-25T14:49:47.569 INFO:teuthology.orchestra.run.smithi083.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-04-25T14:49:47.590 INFO:teuthology.orchestra.run.smithi102.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 838 kB 00:00 2024-04-25T14:49:47.602 INFO:teuthology.orchestra.run.smithi083.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-04-25T14:49:47.636 INFO:teuthology.orchestra.run.smithi083.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-25T14:49:47.678 INFO:teuthology.orchestra.run.smithi083.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 180 kB/s | 20 kB 00:00 2024-04-25T14:49:47.693 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T14:49:47.698 INFO:teuthology.orchestra.run.smithi102.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-04-25T14:49:47.718 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 47 kB/s | 11 kB 00:00 2024-04-25T14:49:47.724 INFO:teuthology.orchestra.run.smithi102.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2024-04-25T14:49:47.736 INFO:teuthology.orchestra.run.smithi083.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T14:49:47.749 INFO:teuthology.orchestra.run.smithi102.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-04-25T14:49:47.760 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 828 kB/s | 160 kB 00:00 2024-04-25T14:49:47.793 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 622 kB/s | 45 kB 00:00 2024-04-25T14:49:47.795 INFO:teuthology.orchestra.run.smithi083.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-04-25T14:49:47.808 INFO:teuthology.orchestra.run.smithi102.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T14:49:47.835 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.7 MB/s | 662 kB 00:00 2024-04-25T14:49:47.846 INFO:teuthology.orchestra.run.smithi083.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-25T14:49:47.850 INFO:teuthology.orchestra.run.smithi102.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T14:49:47.869 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-25T14:49:47.879 INFO:teuthology.orchestra.run.smithi083.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-04-25T14:49:47.894 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-25T14:49:47.913 INFO:teuthology.orchestra.run.smithi083.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T14:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 530 kB/s | 47 kB 00:00 2024-04-25T14:49:47.946 INFO:teuthology.orchestra.run.smithi083.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T14:49:47.975 INFO:teuthology.orchestra.run.smithi102.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-04-25T14:49:47.980 INFO:teuthology.orchestra.run.smithi083.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-04-25T14:49:47.994 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.9 MB/s | 188 kB 00:00 2024-04-25T14:49:48.005 INFO:teuthology.orchestra.run.smithi083.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-25T14:49:48.028 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 292 kB 00:00 2024-04-25T14:49:48.039 INFO:teuthology.orchestra.run.smithi083.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 332 kB/s | 31 kB 00:00 2024-04-25T14:49:48.053 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 287 kB/s | 35 kB 00:00 2024-04-25T14:49:48.098 INFO:teuthology.orchestra.run.smithi083.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T14:49:48.100 INFO:teuthology.orchestra.run.smithi102.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 141 kB/s | 35 kB 00:00 2024-04-25T14:49:48.123 INFO:teuthology.orchestra.run.smithi083.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 503 kB/s | 59 kB 00:00 2024-04-25T14:49:48.125 INFO:teuthology.orchestra.run.smithi102.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-25T14:49:48.140 INFO:teuthology.orchestra.run.smithi083.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-25T14:49:48.151 INFO:teuthology.orchestra.run.smithi102.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 884 kB/s | 303 kB 00:00 2024-04-25T14:49:48.207 INFO:teuthology.orchestra.run.smithi083.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-25T14:49:48.249 INFO:teuthology.orchestra.run.smithi083.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T14:49:48.266 INFO:teuthology.orchestra.run.smithi083.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 714 kB/s | 90 kB 00:00 2024-04-25T14:49:48.268 INFO:teuthology.orchestra.run.smithi102.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T14:49:48.293 INFO:teuthology.orchestra.run.smithi102.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-25T14:49:48.349 INFO:teuthology.orchestra.run.smithi083.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-25T14:49:48.355 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-04-25T14:49:48.375 INFO:teuthology.orchestra.run.smithi083.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-04-25T14:49:48.393 INFO:teuthology.orchestra.run.smithi102.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-25T14:49:48.401 INFO:teuthology.orchestra.run.smithi083.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-25T14:49:48.419 INFO:teuthology.orchestra.run.smithi102.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-25T14:49:48.469 INFO:teuthology.orchestra.run.smithi102.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-04-25T14:49:48.528 INFO:teuthology.orchestra.run.smithi102.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-04-25T14:49:48.553 INFO:teuthology.orchestra.run.smithi102.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-25T14:49:48.558 INFO:teuthology.orchestra.run.smithi083.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T14:49:48.629 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-04-25T14:49:48.645 INFO:teuthology.orchestra.run.smithi102.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-25T14:49:48.670 INFO:teuthology.orchestra.run.smithi102.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2024-04-25T14:49:48.695 INFO:teuthology.orchestra.run.smithi102.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 554 kB/s | 79 kB 00:00 2024-04-25T14:49:48.708 INFO:teuthology.orchestra.run.smithi083.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-25T14:49:48.746 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-25T14:49:48.779 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 2.2 MB/s | 911 kB 00:00 2024-04-25T14:49:48.787 INFO:teuthology.orchestra.run.smithi102.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 406 kB/s | 58 kB 00:00 2024-04-25T14:49:48.804 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 853 kB/s | 133 kB 00:00 2024-04-25T14:49:48.812 INFO:teuthology.orchestra.run.smithi102.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-25T14:49:48.834 INFO:teuthology.orchestra.run.smithi083.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T14:49:48.838 INFO:teuthology.orchestra.run.smithi102.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-25T14:49:48.888 INFO:teuthology.orchestra.run.smithi102.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-04-25T14:49:48.914 INFO:teuthology.orchestra.run.smithi102.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T14:49:48.938 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-25T14:49:48.939 INFO:teuthology.orchestra.run.smithi102.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-25T14:49:48.955 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 583 kB/s | 87 kB 00:00 2024-04-25T14:49:48.998 INFO:teuthology.orchestra.run.smithi102.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T14:49:49.023 INFO:teuthology.orchestra.run.smithi102.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-25T14:49:49.048 INFO:teuthology.orchestra.run.smithi102.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T14:49:49.098 INFO:teuthology.orchestra.run.smithi102.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-25T14:49:49.132 INFO:teuthology.orchestra.run.smithi102.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T14:49:49.157 INFO:teuthology.orchestra.run.smithi102.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-25T14:49:49.189 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T14:49:49.218 INFO:teuthology.orchestra.run.smithi102.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T14:49:49.233 INFO:teuthology.orchestra.run.smithi102.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T14:49:49.243 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 900 kB/s | 244 kB 00:00 2024-04-25T14:49:49.275 INFO:teuthology.orchestra.run.smithi102.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T14:49:49.391 INFO:teuthology.orchestra.run.smithi102.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-25T14:49:49.439 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-25T14:49:49.467 INFO:teuthology.orchestra.run.smithi102.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T14:49:49.473 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 925 kB/s | 248 kB 00:00 2024-04-25T14:49:49.498 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 107 kB/s | 26 kB 00:00 2024-04-25T14:49:49.524 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 238 kB/s | 19 kB 00:00 2024-04-25T14:49:49.549 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 717 kB/s | 35 kB 00:00 2024-04-25T14:49:49.582 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T14:49:49.584 INFO:teuthology.orchestra.run.smithi102.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T14:49:49.674 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-25T14:49:49.825 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T14:49:49.858 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 824 kB/s | 222 kB 00:00 2024-04-25T14:49:49.859 INFO:teuthology.orchestra.run.smithi083.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 3.4 MB 00:01 2024-04-25T14:49:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 395 kB/s | 77 kB 00:00 2024-04-25T14:49:49.909 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 528 kB/s | 42 kB 00:00 2024-04-25T14:49:49.934 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 159 kB 00:00 2024-04-25T14:49:49.967 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-25T14:49:49.992 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.4 MB/s | 279 kB 00:00 2024-04-25T14:49:50.026 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 591 kB/s | 51 kB 00:00 2024-04-25T14:49:50.085 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 592 kB/s | 54 kB 00:00 2024-04-25T14:49:50.127 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-04-25T14:49:50.461 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 123 kB/s | 44 kB 00:00 2024-04-25T14:49:50.511 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 600 kB/s | 221 kB 00:00 2024-04-25T14:49:50.593 INFO:teuthology.orchestra.run.smithi083.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 3.1 MB 00:02 2024-04-25T14:49:50.597 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:49:50.597 INFO:teuthology.orchestra.run.smithi083.stdout:Total 11 MB/s | 222 MB 00:19 2024-04-25T14:49:50.784 INFO:teuthology.orchestra.run.smithi102.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 3.1 MB 00:01 2024-04-25T14:49:50.870 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-25T14:49:50.895 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 85 kB/s | 36 kB 00:00 2024-04-25T14:49:50.920 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 118 kB/s | 47 kB 00:00 2024-04-25T14:49:50.945 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2024-04-25T14:49:50.979 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-25T14:49:51.004 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-25T14:49:51.029 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 804 kB/s | 64 kB 00:00 2024-04-25T14:49:51.063 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 790 kB/s | 45 kB 00:00 2024-04-25T14:49:51.088 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 273 kB/s | 22 kB 00:00 2024-04-25T14:49:51.138 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.9 MB/s | 863 kB 00:00 2024-04-25T14:49:51.376 INFO:teuthology.orchestra.run.smithi102.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.6 MB/s | 3.4 MB 00:02 2024-04-25T14:49:51.379 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:49:51.380 INFO:teuthology.orchestra.run.smithi102.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T14:49:51.597 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-25T14:49:51.665 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-25T14:49:51.764 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 464 kB/s | 309 kB 00:00 2024-04-25T14:49:51.797 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 783 kB/s | 548 kB 00:00 2024-04-25T14:49:51.807 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-25T14:49:51.807 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-25T14:49:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 238 kB/s | 26 kB 00:00 2024-04-25T14:49:51.898 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-04-25T14:49:52.006 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 514 kB/s | 67 kB 00:00 2024-04-25T14:49:52.040 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-25T14:49:52.232 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-25T14:49:52.274 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-25T14:49:52.374 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-04-25T14:49:52.399 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 262 kB/s | 43 kB 00:00 2024-04-25T14:49:52.425 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-04-25T14:49:52.468 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-25T14:49:52.483 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T14:49:52.573 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-25T14:49:52.583 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-25T14:49:52.584 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-25T14:49:52.609 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-25T14:49:52.643 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-25T14:49:52.668 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 372 kB/s | 35 kB 00:00 2024-04-25T14:49:52.710 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T14:49:52.802 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-25T14:49:52.828 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T14:49:52.861 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 755 kB/s | 144 kB 00:00 2024-04-25T14:49:52.886 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2024-04-25T14:49:52.936 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-25T14:49:52.961 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-25T14:49:52.987 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T14:49:53.054 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-25T14:49:53.079 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-25T14:49:53.096 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 728 kB/s | 79 kB 00:00 2024-04-25T14:49:53.138 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 698 kB/s | 58 kB 00:00 2024-04-25T14:49:53.180 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-04-25T14:49:53.214 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T14:49:53.261 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 735 kB/s | 90 kB 00:00 2024-04-25T14:49:53.289 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-04-25T14:49:53.314 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-25T14:49:53.348 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T14:49:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 713 kB/s | 59 kB 00:00 2024-04-25T14:49:53.398 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-04-25T14:49:53.432 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 613 kB/s | 51 kB 00:00 2024-04-25T14:49:53.466 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-25T14:49:53.499 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 900 kB/s | 90 kB 00:00 2024-04-25T14:49:53.541 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-25T14:49:53.566 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-25T14:49:53.617 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T14:49:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-25T14:49:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-25T14:49:53.692 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T14:49:53.801 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-25T14:49:53.968 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T14:49:54.353 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-25T14:49:54.353 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-25T14:49:54.751 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T14:49:55.568 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.7 MB/s | 3.4 MB 00:01 2024-04-25T14:49:55.583 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:49:55.583 INFO:teuthology.orchestra.run.smithi019.stdout:Total 9.8 MB/s | 222 MB 00:22 2024-04-25T14:49:58.439 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T14:49:58.698 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T14:49:58.699 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T14:49:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-25T14:49:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:49:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:49:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:49:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:49:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:49:59.882 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:50:00.208 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:50:00.285 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-25T14:50:00.324 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:00.447 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:50:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:50:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:50:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:50:00.761 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:50:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:00.823 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:50:00.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:50:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:00.898 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:50:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:00.941 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:50:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:01.278 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:50:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:50:01.391 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:01.398 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T14:50:01.399 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T14:50:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:50:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:50:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:50:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:01.542 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:50:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:50:01.633 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:50:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:50:01.683 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:50:01.711 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:50:01.779 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:50:01.815 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:50:01.832 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:50:01.859 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:50:01.863 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:50:01.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:01.899 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:50:01.974 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:50:01.982 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:02.005 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:02.009 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:50:02.053 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:50:02.068 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:02.082 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:50:02.090 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:02.123 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:02.151 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:50:02.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:50:02.271 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:50:02.323 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:02.356 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:02.413 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:50:02.462 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:50:02.465 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:50:02.558 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:50:02.597 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:50:02.632 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:50:02.696 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:50:02.731 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:50:02.774 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:50:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:50:02.835 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:50:02.903 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:50:02.957 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:50:02.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:50:02.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:50:03.023 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:50:03.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:50:03.099 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:50:03.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:50:03.134 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:50:03.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:50:03.177 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:50:03.179 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:50:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:50:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:50:03.274 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:50:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:50:03.388 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:50:03.402 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:50:03.433 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:50:03.488 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:50:03.535 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:50:03.569 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:50:03.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:50:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:50:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:50:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:50:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:50:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:50:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:50:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:50:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:50:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:50:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:50:04.144 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:50:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:50:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:50:04.237 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:50:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:50:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:50:04.265 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:50:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:50:04.308 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:50:04.346 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:50:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:50:04.388 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:50:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:50:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:50:04.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:50:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:50:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:50:04.599 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:50:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:50:04.654 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:50:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:50:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:50:04.727 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:50:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:50:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:50:04.766 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:50:04.803 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:50:04.811 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:50:04.838 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:50:04.858 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:50:04.893 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:50:04.925 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:50:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:50:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:50:04.979 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:50:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:50:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:50:05.115 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:50:05.153 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:50:05.159 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:50:05.190 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:50:05.201 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:50:05.231 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:50:05.324 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:50:05.356 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:50:05.397 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:50:05.422 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:50:05.447 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:50:05.521 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:50:05.648 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:50:05.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:50:05.715 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:50:05.750 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:50:05.789 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:50:05.829 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:50:05.859 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:50:05.893 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:50:05.921 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:50:05.926 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:50:05.965 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:50:05.968 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:50:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:50:06.003 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:50:06.088 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:50:06.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:50:06.141 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:50:06.148 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:50:06.172 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:50:06.220 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:50:06.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:50:06.293 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:50:06.318 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:50:06.365 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:50:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:50:06.499 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:50:06.576 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:50:07.087 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:50:07.151 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:50:07.180 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:50:07.223 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:50:07.278 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:50:07.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:50:07.339 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:50:07.450 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:50:07.491 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:50:07.667 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:50:07.697 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:50:07.774 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:50:08.015 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T14:50:08.117 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:50:08.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:50:08.307 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:50:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:50:08.409 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:50:08.412 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:50:08.520 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:50:08.525 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:50:08.561 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:50:08.654 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:50:08.682 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:50:08.716 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:50:08.974 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:50:09.026 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:50:09.062 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:09.151 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:09.183 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:50:09.219 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:50:09.297 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:09.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:09.520 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:09.567 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:50:09.698 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:50:09.729 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:50:09.732 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:09.818 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:50:09.819 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:09.841 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:50:09.846 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:50:09.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:50:09.984 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:50:10.092 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:50:10.129 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:10.190 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:50:10.225 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:10.298 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:10.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:50:10.366 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:50:10.369 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:10.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:50:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:10.654 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:50:10.843 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:50:10.870 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:10.974 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:10.997 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:50:11.083 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:11.142 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:50:11.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:50:11.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:50:11.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:50:11.589 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:50:11.737 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:50:11.787 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:50:11.853 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:50:11.900 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:50:11.939 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:50:12.059 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:50:12.099 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:12.130 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:12.152 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:50:12.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:50:12.251 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:50:12.370 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:50:12.411 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:50:12.479 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:50:12.518 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:50:12.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:50:12.674 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:12.705 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:50:12.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:12.812 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:12.812 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:50:12.813 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:50:12.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:12.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:50:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:13.002 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:50:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:50:13.268 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:13.310 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:13.804 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:50:13.845 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:13.889 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:13.915 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:50:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:50:13.989 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:50:13.989 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:50:13.989 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:50:14.023 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:50:14.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:50:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:50:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:50:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:50:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:50:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:50:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:50:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:50:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:50:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:50:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:50:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:50:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:50:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:50:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:50:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:50:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:50:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:50:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:50:15.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:50:15.869 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:50:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:50:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:50:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:50:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:50:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:50:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:50:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:50:16.447 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:50:16.496 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:50:16.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:50:16.628 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:50:16.787 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:50:16.851 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:50:16.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:50:17.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:50:17.162 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:50:17.229 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:50:18.225 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:50:18.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:50:18.345 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:50:18.469 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:50:18.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:50:18.653 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:50:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:50:18.988 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:50:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:50:19.143 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:50:20.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:50:20.203 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:50:21.862 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:50:21.965 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:50:22.006 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:50:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:50:22.455 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:50:22.511 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:22.686 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:50:22.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:50:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:50:23.392 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:50:23.430 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:23.556 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:50:23.582 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:50:24.082 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:24.082 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-25T14:50:24.082 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-25T14:50:24.082 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-25T14:50:24.083 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-25T14:50:24.083 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-25T14:50:24.083 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-25T14:50:24.083 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-25T14:50:24.083 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:24.256 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:24.256 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:50:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:25.635 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-04-25T14:50:25.636 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:25.744 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:25.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:25.794 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:25.809 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:25.809 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:25.809 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:50:25.809 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:25.995 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:26.017 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:26.462 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:26.484 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:26.484 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:26.484 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:50:26.484 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:26.485 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:26.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:26.593 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:50:26.618 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:26.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:50:26.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:26.696 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:50:26.880 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:50:26.930 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:50:27.007 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:50:27.095 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:50:27.226 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:50:27.271 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:50:27.288 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:27.304 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:50:27.319 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:27.319 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:27.319 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:50:27.319 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:27.320 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:27.320 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:27.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:50:27.371 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:50:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:27.548 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:27.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:27.625 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:50:27.625 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:50:27.625 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:50:27.625 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:27.676 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:27.809 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:27.839 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:27.871 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:50:27.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:50:27.970 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:28.025 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:28.047 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:28.146 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:50:28.195 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:50:28.250 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:50:28.433 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:50:28.484 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:50:28.562 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:50:28.650 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:50:28.789 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:50:28.826 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:50:28.866 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:50:28.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:50:29.000 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:29.056 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:29.364 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:29.393 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:29.425 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:50:29.486 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:50:29.516 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:30.014 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:30.048 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:30.149 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:30.321 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:50:30.350 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:30.373 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:30.373 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:30.374 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:50:30.374 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:30.374 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:30.374 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:30.417 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:30.428 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:30.525 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:50:30.552 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:30.573 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:50:30.584 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:31.492 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:50:31.528 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:31.549 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:31.550 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:31.550 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:31.550 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:50:31.551 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:31.551 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:31.551 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:31.576 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:31.595 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:31.617 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:31.617 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:31.617 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:50:31.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:31.662 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:31.677 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:31.685 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:31.839 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:50:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:31.902 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:31.902 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:50:31.902 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:31.902 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:31.946 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:31.957 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:32.011 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:32.011 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:50:32.011 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:32.012 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:32.061 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:50:32.110 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:50:33.033 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:50:33.065 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:33.087 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:33.087 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:33.087 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:50:33.087 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:33.087 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:33.088 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:33.132 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:33.154 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:33.154 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:33.154 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:50:33.154 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:33.199 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:33.221 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:33.222 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:33.222 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:50:33.222 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:33.222 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:33.222 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:33.517 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:36.576 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:50:36.617 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:50:36.657 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:50:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:50:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:50:36.899 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:50:36.938 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:50:36.969 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:50:37.597 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:37.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:37.660 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:38.113 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:50:38.153 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:50:38.194 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:50:38.226 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:50:38.254 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:50:38.437 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:50:38.478 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:50:38.492 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:50:38.522 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:50:38.571 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:50:38.612 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:50:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:50:38.751 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:50:38.751 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:50:38.835 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:50:38.868 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:50:38.868 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:50:38.963 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:50:39.032 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:39.142 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:39.186 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:39.206 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:40.034 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:50:40.125 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:50:40.166 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:50:40.247 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:50:40.279 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:50:40.279 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:50:40.363 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:50:40.396 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:50:40.396 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:50:40.484 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:50:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:41.242 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-25T14:50:41.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:50:41.526 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:41.526 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:50:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:43.231 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:43.321 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:50:43.321 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:43.321 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:50:43.321 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:43.322 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:50:43.322 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:43.590 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:43.663 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:50:43.664 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:43.664 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:50:43.664 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:43.664 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:50:43.664 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:44.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:50:44.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:44.401 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:50:44.465 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:50:44.794 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:50:44.855 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:50:44.940 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:50:44.941 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:50:44.942 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:50:44.943 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:50:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:50:44.945 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:50:44.946 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:50:44.947 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:50:44.948 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:50:44.949 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:50:44.950 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:50:44.951 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:50:44.952 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:50:45.058 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:50:45.239 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:50:45.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:50:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:50:45.384 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:50:46.135 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:46.181 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:50:46.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:50:46.347 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:50:46.402 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:50:46.403 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:50:46.404 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:50:46.416 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:50:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:50:46.418 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:50:46.419 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:50:46.420 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:50:46.421 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:50:46.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:50:46.423 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:50:46.424 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:50:46.425 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-25T14:50:47.369 DEBUG:teuthology.parallel:result is None 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-25T14:50:48.938 DEBUG:teuthology.parallel:result is None 2024-04-25T14:50:48.998 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:50:49.025 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:49.177 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:50:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:50:49.393 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:50:49.471 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:49.544 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:49.582 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:50:49.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:50:49.797 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:50:50.892 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:50:50.939 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:51.009 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:50:51.009 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:51.009 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:50:51.010 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:51.010 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:50:51.010 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:51.073 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:51.145 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:50:51.145 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:51.145 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:50:51.145 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:51.206 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:50:51.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:51.682 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:50:51.758 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:50:57.239 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:50:57.311 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:50:57.402 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:50:57.452 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:50:57.500 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:50:57.851 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:50:58.012 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:50:58.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:50:59.133 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:50:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:51:00.740 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:51:00.894 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:51:01.019 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:51:01.212 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:51:01.259 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:51:01.259 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:51:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:51:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:51:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:51:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:51:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:51:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:51:02.124 INFO:teuthology.orchestra.run.smithi019.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T14:51:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:51:08.130 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:51:08.130 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:51:08.130 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:51:08.130 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:51:08.130 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:51:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:51:08.132 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:51:08.133 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:51:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:51:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:51:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:51:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:51:08.138 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:51:08.139 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:51:08.140 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:51:08.141 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:51:08.142 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:51:08.143 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:51:08.144 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:51:08.144 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:51:08.144 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:51:08.144 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:51:10.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:51:10.852 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.853 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.854 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.854 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.854 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.854 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.854 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.855 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:51:10.856 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.857 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:51:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:51:10.859 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:51:10.860 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:51:10.861 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:51:10.862 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:51:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.864 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:51:10.865 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T14:51:11.454 DEBUG:teuthology.parallel:result is None 2024-04-25T14:51:11.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:11.681 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:51:11.750 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:11.751 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:11.751 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:51:11.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:11.891 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:51:11.917 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:11.917 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:11.918 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:51:11.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:12.151 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:51:12.176 INFO:teuthology.orchestra.run.smithi102.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:12.176 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:51:12.176 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:51:12.178 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T14:51:12.178 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:51:12.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:51:12.245 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:51:12.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:51:12.276 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:51:12.277 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:51:12.306 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T14:51:12.306 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:51:12.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:51:12.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:51:12.477 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:51:12.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:51:12.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:51:12.574 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:51:12.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:51:12.602 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:51:12.668 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T14:51:12.668 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:51:12.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:51:12.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:51:12.843 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:51:12.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:51:12.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:51:12.930 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:51:12.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:51:12.955 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:51:13.018 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T14:51:13.019 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:51:13.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:51:13.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:51:13.185 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:51:13.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:51:13.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:51:13.272 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:51:13.272 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:51:13.296 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:51:13.359 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T14:51:13.466 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T14:51:13.466 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:13.467 INFO:tasks.cephadm:Cluster fsid is 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:51:13.467 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T14:51:13.467 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.83', 'mon.c': '172.21.15.102'} 2024-04-25T14:51:13.467 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-04-25T14:51:13.467 INFO:tasks.cephadm:First mgr is x 2024-04-25T14:51:13.467 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T14:51:13.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-04-25T14:51:13.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-25T14:51:13.564 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-04-25T14:51:13.588 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T14:51:13.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:13.806 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T14:51:13.945 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:13.947 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T14:51:13.947 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:14.417 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:51 /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:14.417 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:14.844 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:51 /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:14.845 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:15.247 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:51 /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:15.247 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:15.295 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:15.314 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:51:15.348 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T14:51:15.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:51:15.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:51:15.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:51:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:51:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:51:56.575 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-25T14:51:56.576 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-25T14:52:21.855 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T14:52:21.856 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T14:52:25.005 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T14:52:25.006 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T14:52:25.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-04-25T14:52:25.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-25T14:52:25.178 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-04-25T14:52:25.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-04-25T14:52:25.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-25T14:52:25.325 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-04-25T14:52:25.352 INFO:tasks.cephadm:Writing seed config... 2024-04-25T14:52:25.353 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T14:52:25.353 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T14:52:25.354 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T14:52:25.355 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T14:52:25.356 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T14:52:25.357 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:52:25.357 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T14:52:25.416 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 42f950da-0313-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T14:52:25.416 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T14:52:25.463 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T14:52:25.508 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T14:52:25.509 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 42f950da-0313-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:25.794 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:52:25.794 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '42f950da-0313-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-04-25T14:52:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T14:52:25.795 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-04-25T14:52:25.857 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:52:25.857 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-04-25T14:52:25.857 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-04-25T14:52:25.879 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:52:25.879 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:52:25.899 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:52:25.899 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:52:25.925 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T14:52:25.947 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T14:52:25.947 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:52:25.947 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-04-25T14:52:26.019 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:52:26.020 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T14:52:26.020 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-04-25T14:52:26.020 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-04-25T14:52:26.040 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:52:26.040 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:52:26.062 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:52:26.062 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:52:26.088 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T14:52:26.111 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T14:52:26.111 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:52:26.111 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-04-25T14:52:26.111 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:26.112 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140642981461152 on /run/cephadm/42f950da-0313-11ef-bc93-c7b262605968.lock 2024-04-25T14:52:26.112 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140642981461152 acquired on /run/cephadm/42f950da-0313-11ef-bc93-c7b262605968.lock 2024-04-25T14:52:26.113 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-04-25T14:52:26.115 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-04-25T14:52:26.115 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T14:52:26.129 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-04-25T14:52:26.129 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-04-25T14:52:26.142 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T14:52:26.142 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T14:52:26.142 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T14:52:26.154 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T14:52:26.154 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T14:52:26.155 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:52:26.155 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T14:52:26.155 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T14:52:26.155 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:52:26.158 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T14:52:26.158 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T14:52:26.158 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T14:52:26.161 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T14:52:26.164 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:52:27.826 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T14:52:29.073 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:52:29.073 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:52:29.073 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T14:52:30.259 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-04-25T14:52:30.260 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-04-25T14:52:31.359 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAubipmms9uKhAAJKtn0h3qPCLUfNQkL0pjLg== 2024-04-25T14:52:32.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAvbipm/pFTNBAAYhmHpY8WPOwQ9jS44O7Ysw== 2024-04-25T14:52:33.748 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAxbipmdkqfABAAeePPr6UZ5fm0WjH1s7U6LA== 2024-04-25T14:52:33.751 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:52:34.901 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:52:34.902 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-04-25T14:52:36.116 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-04-25T14:52:36.525 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T14:52:36.779 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T14:52:37.100 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-42f950da-0313-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968.target. 2024-04-25T14:52:37.100 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-42f950da-0313-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968.target. 2024-04-25T14:52:37.529 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:52:37.529 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T14:52:37.837 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968.target.wants/ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968@.service. 2024-04-25T14:52:38.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 podman[27702]: 2024-04-25 14:52:38.287112444 +0000 UTC m=+0.151477243 container create f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T14:52:38.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 podman[27702]: 2024-04-25 14:52:38.533786047 +0000 UTC m=+0.398150878 container init f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:52:38.676 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:52:38.676 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T14:52:38.695 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:52:38.695 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:52:38.695 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T14:52:38.695 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:52:38.696 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-04-25T14:52:38.696 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-04-25T14:52:38.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 podman[27702]: 2024-04-25 14:52:38.544867529 +0000 UTC m=+0.409232326 container start f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T14:52:38.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 ceph-mon[27720]: mkfs 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:38.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 ceph-mon[27720]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:52:38.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 bash[27702]: f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 2024-04-25T14:52:38.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:38 smithi019 systemd[1]: Started Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: monmap epoch 1 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: last_changed 2024-04-25T14:52:34.247134+0000 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: min_mon_release 19 (squid) 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: election_strategy: 1 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: fsmap 2024-04-25T14:52:39.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:52:39.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: mgrmap e1: no daemons active 2024-04-25T14:52:39.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:39 smithi019 ceph-mon[27720]: from='client.? 172.21.15.19:0/1366661219' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T14:52:40.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T14:52:40.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:40.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T14:52:40.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:40.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-04-25T14:52:40.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.807269s) 2024-04-25T14:52:40.045 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T14:52:40.045 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T14:52:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-04-25T14:52:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T14:52:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T14:52:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T14:52:40.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T14:52:40.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:40.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.336+0000 7fcc51800700 1 Processor -- start 2024-04-25T14:52:40.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.336+0000 7fcc51800700 1 -- start start 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.337+0000 7fcc51800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.337+0000 7fcc51800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1109a0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.338+0000 7fcc4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.338+0000 7fcc4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54932/0 (socket says 172.21.15.19:54932) 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.338+0000 7fcc4be00700 1 -- 172.21.15.19:0/1795944559 learned_addr learned my addr 172.21.15.19:0/1795944559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.338+0000 7fcc4be00700 1 -- 172.21.15.19:0/1795944559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c10ecb0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.339+0000 7fcc4be00700 1 --2- 172.21.15.19:0/1795944559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098a0 tx=0x7fcc3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91226afe90b89d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.339+0000 7fcc4aa00700 1 -- 172.21.15.19:0/1795944559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c004160 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.339+0000 7fcc4aa00700 1 -- 172.21.15.19:0/1795944559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c004300 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.339+0000 7fcc4aa00700 1 -- 172.21.15.19:0/1795944559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c0036a0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 -- 172.21.15.19:0/1795944559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 msgr2=0x7fcc4c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 --2- 172.21.15.19:0/1795944559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098a0 tx=0x7fcc3c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 -- 172.21.15.19:0/1795944559 shutdown_connections 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 --2- 172.21.15.19:0/1795944559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1103d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 -- 172.21.15.19:0/1795944559 >> 172.21.15.19:0/1795944559 conn(0x7fcc4c07dbf0 msgr2=0x7fcc4c080010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 -- 172.21.15.19:0/1795944559 shutdown_connections 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.340+0000 7fcc51800700 1 -- 172.21.15.19:0/1795944559 wait complete. 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.341+0000 7fcc51800700 1 Processor -- start 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.341+0000 7fcc51800700 1 -- start start 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.342+0000 7fcc51800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.342+0000 7fcc51800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10f1e0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.342+0000 7fcc4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.342+0000 7fcc4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54938/0 (socket says 172.21.15.19:54938) 2024-04-25T14:52:40.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.342+0000 7fcc4be00700 1 -- 172.21.15.19:0/1366661219 learned_addr learned my addr 172.21.15.19:0/1366661219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc4be00700 1 -- 172.21.15.19:0/1366661219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c1a25b0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc4be00700 1 --2- 172.21.15.19:0/1366661219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c082120 tx=0x7fcc3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c003db0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c011630 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c1a2840 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc3c011930 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.343+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a5530 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.344+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc3c011ad0 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.344+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc3c01b450 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.345+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c069b80 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.348+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc3c004260 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.430+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fcc4c02e080 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.431+0000 7fcc48c00700 1 -- 172.21.15.19:0/1366661219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fcc3c025480 con 0x7fcc4c10ffb0 2024-04-25T14:52:40.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 msgr2=0x7fcc4c1a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 --2- 172.21.15.19:0/1366661219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c082120 tx=0x7fcc3c004750 comp rx=0 tx=0).stop 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 shutdown_connections 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 --2- 172.21.15.19:0/1366661219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc4c10ffb0 0x7fcc4c1a2070 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 >> 172.21.15.19:0/1366661219 conn(0x7fcc4c07dbf0 msgr2=0x7fcc4c198d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 shutdown_connections 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:39.432+0000 7fcc51800700 1 -- 172.21.15.19:0/1366661219 wait complete. 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-04-25T14:52:40.056 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T14:52:41.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:40 smithi019 ceph-mon[27720]: from='client.? 172.21.15.19:0/3799051664' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:52:41.084 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:40 smithi019 ceph-mon[27720]: from='client.? 172.21.15.19:0/3799051664' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:52:41.383 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:52:41.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:52:41.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:52:41.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.644+0000 7f3c6bc00700 1 Processor -- start 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.645+0000 7f3c6bc00700 1 -- start start 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.645+0000 7f3c6bc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.645+0000 7f3c6bc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6410fa30 con 0x7f3c6410f040 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.646+0000 7f3c69200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.646+0000 7f3c69200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54940/0 (socket says 172.21.15.19:54940) 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.646+0000 7f3c69200700 1 -- 172.21.15.19:0/632104664 learned_addr learned my addr 172.21.15.19:0/632104664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.646+0000 7f3c69200700 1 -- 172.21.15.19:0/632104664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6410fdc0 con 0x7f3c6410f040 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.647+0000 7f3c69200700 1 --2- 172.21.15.19:0/632104664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c580098a0 tx=0x7f3c58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96dfc3541dcc14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:41.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.647+0000 7f3c63400700 1 -- 172.21.15.19:0/632104664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c58004160 con 0x7f3c6410f040 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.647+0000 7f3c63400700 1 -- 172.21.15.19:0/632104664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c58004300 con 0x7f3c6410f040 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.647+0000 7f3c63400700 1 -- 172.21.15.19:0/632104664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c580036a0 con 0x7f3c6410f040 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 -- 172.21.15.19:0/632104664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 msgr2=0x7f3c6410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 --2- 172.21.15.19:0/632104664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c580098a0 tx=0x7f3c58009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 -- 172.21.15.19:0/632104664 shutdown_connections 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 --2- 172.21.15.19:0/632104664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c6410f460 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 -- 172.21.15.19:0/632104664 >> 172.21.15.19:0/632104664 conn(0x7f3c6407ddf0 msgr2=0x7f3c64080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 -- 172.21.15.19:0/632104664 shutdown_connections 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.648+0000 7f3c6bc00700 1 -- 172.21.15.19:0/632104664 wait complete. 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.649+0000 7f3c6bc00700 1 Processor -- start 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.650+0000 7f3c6bc00700 1 -- start start 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.650+0000 7f3c6bc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.650+0000 7f3c6bc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c641102f0 con 0x7f3c6410f040 2024-04-25T14:52:41.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.651+0000 7f3c69200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.651+0000 7f3c69200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54954/0 (socket says 172.21.15.19:54954) 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.651+0000 7f3c69200700 1 -- 172.21.15.19:0/3799051664 learned_addr learned my addr 172.21.15.19:0/3799051664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.651+0000 7f3c69200700 1 -- 172.21.15.19:0/3799051664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c641a2730 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c69200700 1 --2- 172.21.15.19:0/3799051664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c58003db0 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c58011630 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c58011950 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c641a29c0 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.652+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c641a56b0 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.653+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c58011af0 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.653+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c5801c450 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.654+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c640640a0 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.657+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3c58032020 con 0x7f3c6410f040 2024-04-25T14:52:41.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.734+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3c6402e080 con 0x7f3c6410f040 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.740+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f3c58004260 con 0x7f3c6410f040 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.741+0000 7f3c61600700 1 -- 172.21.15.19:0/3799051664 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c5801c650 con 0x7f3c6410f040 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 msgr2=0x7f3c641a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 --2- 172.21.15.19:0/3799051664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c58004750 comp rx=0 tx=0).stop 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 shutdown_connections 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 --2- 172.21.15.19:0/3799051664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c6410f040 0x7f3c641a21f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 >> 172.21.15.19:0/3799051664 conn(0x7f3c6407ddf0 msgr2=0x7f3c64198ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.742+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 shutdown_connections 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:40.743+0000 7f3c6bc00700 1 -- 172.21.15.19:0/3799051664 wait complete. 2024-04-25T14:52:41.393 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-04-25T14:52:42.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:42 smithi019 ceph-mon[27720]: from='client.? 172.21.15.19:0/288871710' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:42.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.990+0000 7f6e19000700 1 Processor -- start 2024-04-25T14:52:42.711 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.991+0000 7f6e19000700 1 -- start start 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.991+0000 7f6e19000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.991+0000 7f6e19000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14063d60 con 0x7f6e14109040 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.992+0000 7f6e11e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.992+0000 7f6e11e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54956/0 (socket says 172.21.15.19:54956) 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.992+0000 7f6e11e00700 1 -- 172.21.15.19:0/2988782195 learned_addr learned my addr 172.21.15.19:0/2988782195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.992+0000 7f6e11e00700 1 -- 172.21.15.19:0/2988782195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1410b970 con 0x7f6e14109040 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e11e00700 1 --2- 172.21.15.19:0/2988782195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098a0 tx=0x7f6e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa79da6ec237d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e10a00700 1 -- 172.21.15.19:0/2988782195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e04004160 con 0x7f6e14109040 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e10a00700 1 -- 172.21.15.19:0/2988782195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e04004300 con 0x7f6e14109040 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e10a00700 1 -- 172.21.15.19:0/2988782195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e04011af0 con 0x7f6e14109040 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e19000700 1 -- 172.21.15.19:0/2988782195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 msgr2=0x7f6e1410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:42.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.993+0000 7f6e19000700 1 --2- 172.21.15.19:0/2988782195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098a0 tx=0x7f6e04009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.994+0000 7f6e19000700 1 -- 172.21.15.19:0/2988782195 shutdown_connections 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.994+0000 7f6e19000700 1 --2- 172.21.15.19:0/2988782195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e1410b430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.994+0000 7f6e19000700 1 -- 172.21.15.19:0/2988782195 >> 172.21.15.19:0/2988782195 conn(0x7f6e14102ab0 msgr2=0x7f6e14104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.994+0000 7f6e19000700 1 -- 172.21.15.19:0/2988782195 shutdown_connections 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.994+0000 7f6e19000700 1 -- 172.21.15.19:0/2988782195 wait complete. 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.995+0000 7f6e19000700 1 Processor -- start 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.995+0000 7f6e19000700 1 -- start start 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.995+0000 7f6e19000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.995+0000 7f6e19000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1410c8d0 con 0x7f6e14109040 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.996+0000 7f6e11e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:42.716 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.996+0000 7f6e11e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54972/0 (socket says 172.21.15.19:54972) 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.996+0000 7f6e11e00700 1 -- 172.21.15.19:0/288871710 learned_addr learned my addr 172.21.15.19:0/288871710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.996+0000 7f6e11e00700 1 -- 172.21.15.19:0/288871710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e14108e00 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.996+0000 7f6e11e00700 1 --2- 172.21.15.19:0/288871710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e04011ac0 tx=0x7f6e04011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e04003b20 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e0401a8c0 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e040248d0 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e14106ff0 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e14107530 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e04022020 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.997+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e0402dde0 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:41.998+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e140640a0 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.001+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6e04042e50 con 0x7f6e14109040 2024-04-25T14:52:42.717 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.079+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6e14107b60 con 0x7f6e14109040 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.079+0000 7f6e0aa00700 1 -- 172.21.15.19:0/288871710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6e04031020 con 0x7f6e14109040 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 msgr2=0x7f6e141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 --2- 172.21.15.19:0/288871710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e04011ac0 tx=0x7f6e04011ba0 comp rx=0 tx=0).stop 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 shutdown_connections 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 --2- 172.21.15.19:0/288871710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e14109040 0x7f6e141088c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 >> 172.21.15.19:0/288871710 conn(0x7f6e14102ab0 msgr2=0x7f6e14103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 shutdown_connections 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:42.081+0000 7f6e19000700 1 -- 172.21.15.19:0/288871710 wait complete. 2024-04-25T14:52:42.718 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-04-25T14:52:43.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:42 smithi019 systemd[1]: Stopping Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T14:52:43.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a[27716]: 2024-04-25T14:52:43.244+0000 7f859b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a[27716]: 2024-04-25T14:52:43.244+0000 7f859b600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T14:52:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:43 smithi019 podman[27964]: 2024-04-25 14:52:43.348456139 +0000 UTC m=+0.255287670 container died f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:44.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:43 smithi019 podman[27964]: 2024-04-25 14:52:43.75184981 +0000 UTC m=+0.658681334 container cleanup f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:52:44.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:43 smithi019 bash[27964]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:52:50.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:50 smithi019 podman[27978]: 2024-04-25 14:52:50.561469554 +0000 UTC m=+7.192227003 container remove f979515f45ecd8e99de0b349be2caffe01c1d62be414cf77e2a14fdcccf349b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:52:51.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:51 smithi019 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T14:52:51.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:51 smithi019 systemd[1]: Stopped Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:52:51.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:51 smithi019 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service: Consumed 2.062s CPU time. 2024-04-25T14:52:51.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:51 smithi019 systemd[1]: Starting Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T14:52:51.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:51 smithi019 podman[28071]: 2024-04-25 14:52:51.522050713 +0000 UTC m=+0.037286925 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:52:52.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:52 smithi019 podman[28071]: 2024-04-25 14:52:52.696403255 +0000 UTC m=+1.211639504 container create 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T14:52:53.076 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:52 smithi019 podman[28071]: 2024-04-25 14:52:52.991610973 +0000 UTC m=+1.506847228 container init 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 podman[28071]: 2024-04-25 14:52:53.001708155 +0000 UTC m=+1.516944393 container start 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: pidfile_write: ignore empty --pid-file 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: load: jerasure load: lrc 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Git sha 0 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:52:53.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: DB SUMMARY 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: DB Session ID: 0Y2V4A4MH3HXAX4MQPW8 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104886 ; 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.env: 0x55b197083cc0 2024-04-25T14:52:53.359 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.info_log: 0x55b1d6d47c40 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.statistics: (nil) 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.use_fsync: 0 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:52:53.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.db_log_dir: 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.wal_dir: 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:52:53.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.write_buffer_manager: 0x55b1d6cdb0e0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.unordered_write: 0 2024-04-25T14:52:53.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:52:53.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:52:53.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:52:53.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:52:53.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.row_cache: None 2024-04-25T14:52:53.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.wal_filter: None 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.wal_compression: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:52:53.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:52:53.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_open_files: -1 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Compression algorithms supported: 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kZSTD supported: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:52:53.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.merge_operator: 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_filter: None 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:52:53.367 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b1d6d47e00) 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55b1d6d0d1f0 2024-04-25T14:52:53.368 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-04-25T14:52:53.369 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-04-25T14:52:53.370 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression: NoCompression 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.num_levels: 7 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:52:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:52:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:52:53.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:52:53.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:52:53.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:52:53.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:52:53.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:52:53.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:52:53.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.ttl: 2592000 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:52:53.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c5547843-3797-4ca3-bd46-4d43fc379853 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056773060998, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056773062857, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97916, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90260, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714056773, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c5547843-3797-4ca3-bd46-4d43fc379853", "db_session_id": "0Y2V4A4MH3HXAX4MQPW8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056773063032, "job": 1, "event": "recovery_finished"} 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 bash[28071]: 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:52:53.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b1d6e22000 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: DB pointer 0x55b1d6e0c000 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T14:52:53.379 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:52:53.380 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 13.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 13.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55b1d6d0d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???) e1 preinit fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:53.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).mds e1 new map 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).mds e1 print_map 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 systemd[1]: Started Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:52:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: monmap epoch 1 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: last_changed 2024-04-25T14:52:34.247134+0000 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: min_mon_release 19 (squid) 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: election_strategy: 1 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: fsmap 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:52:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:53 smithi019 ceph-mon[28090]: mgrmap e1: no daemons active 2024-04-25T14:52:54.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.667+0000 7fbfebe00700 1 Processor -- start 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.668+0000 7fbfebe00700 1 -- start start 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.668+0000 7fbfebe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.668+0000 7fbfebe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4063cc0 con 0x7fbfe4106ed0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.669+0000 7fbfe9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.669+0000 7fbfe9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32782/0 (socket says 172.21.15.19:32782) 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.669+0000 7fbfe9400700 1 -- 172.21.15.19:0/2851713353 learned_addr learned my addr 172.21.15.19:0/2851713353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.669+0000 7fbfe9400700 1 -- 172.21.15.19:0/2851713353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe4107830 con 0x7fbfe4106ed0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.670+0000 7fbfe9400700 1 --2- 172.21.15.19:0/2851713353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009e80 tx=0x7fbfd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c11883195d7d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.670+0000 7fbfe3e00700 1 -- 172.21.15.19:0/2851713353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8017070 con 0x7fbfe4106ed0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.670+0000 7fbfe3e00700 1 -- 172.21.15.19:0/2851713353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbfd800b8c0 con 0x7fbfe4106ed0 2024-04-25T14:52:54.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.670+0000 7fbfe3e00700 1 -- 172.21.15.19:0/2851713353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8004490 con 0x7fbfe4106ed0 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 -- 172.21.15.19:0/2851713353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 msgr2=0x7fbfe41072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 --2- 172.21.15.19:0/2851713353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009e80 tx=0x7fbfd800b330 comp rx=0 tx=0).stop 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 -- 172.21.15.19:0/2851713353 shutdown_connections 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 --2- 172.21.15.19:0/2851713353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe41072f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 -- 172.21.15.19:0/2851713353 >> 172.21.15.19:0/2851713353 conn(0x7fbfe4102ab0 msgr2=0x7fbfe4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 -- 172.21.15.19:0/2851713353 shutdown_connections 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.671+0000 7fbfebe00700 1 -- 172.21.15.19:0/2851713353 wait complete. 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.673+0000 7fbfebe00700 1 Processor -- start 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.673+0000 7fbfebe00700 1 -- start start 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.673+0000 7fbfebe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.673+0000 7fbfebe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41087f0 con 0x7fbfe4106ed0 2024-04-25T14:52:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.674+0000 7fbfe9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.674+0000 7fbfe9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32796/0 (socket says 172.21.15.19:32796) 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.674+0000 7fbfe9400700 1 -- 172.21.15.19:0/3414037253 learned_addr learned my addr 172.21.15.19:0/3414037253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.674+0000 7fbfe9400700 1 -- 172.21.15.19:0/3414037253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe419eb90 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfe9400700 1 --2- 172.21.15.19:0/3414037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b7b0 tx=0x7fbfd800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8020020 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe419ee20 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbfd8003d10 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8017040 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a1b10 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.675+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbfd8023020 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.676+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbfd800f050 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.676+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe40640a0 con 0x7fbfe4106ed0 2024-04-25T14:52:54.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.680+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbfd802a020 con 0x7fbfe4106ed0 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.759+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbfe402e170 con 0x7fbfe4106ed0 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.763+0000 7fbfe2000700 1 -- 172.21.15.19:0/3414037253 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbfd8003ec0 con 0x7fbfe4106ed0 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 msgr2=0x7fbfe419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 --2- 172.21.15.19:0/3414037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b7b0 tx=0x7fbfd800be90 comp rx=0 tx=0).stop 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 shutdown_connections 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 --2- 172.21.15.19:0/3414037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe4106ed0 0x7fbfe419e650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 >> 172.21.15.19:0/3414037253 conn(0x7fbfe4102ab0 msgr2=0x7fbfe41952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 shutdown_connections 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:53.765+0000 7fbfebe00700 1 -- 172.21.15.19:0/3414037253 wait complete. 2024-04-25T14:52:54.487 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T14:52:54.503 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:54.504 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-04-25T14:52:54.504 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T14:52:54.505 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T14:52:55.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:54 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3414037253' entity='client.admin' 2024-04-25T14:52:55.209 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968.target.wants/ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-42f950da-0313-11ef-bc93-c7b262605968@.service. 2024-04-25T14:52:55.371 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 systemd[1]: Starting Ceph mgr.x for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T14:52:55.861 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 podman[28311]: 2024-04-25 14:52:55.505570803 +0000 UTC m=+0.032184017 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:52:55.861 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 podman[28311]: 2024-04-25 14:52:55.623373381 +0000 UTC m=+0.149986606 container create 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:52:55.862 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 podman[28311]: 2024-04-25 14:52:55.860420749 +0000 UTC m=+0.387034040 container init 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T14:52:55.973 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:52:55.974 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T14:52:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:52:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:52:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T14:52:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:52:56.015 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:52:56.015 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T14:52:56.036 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:52:56.036 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:52:56.037 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T14:52:56.037 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T14:52:56.037 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-04-25T14:52:56.037 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-04-25T14:52:56.115 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 podman[28311]: 2024-04-25 14:52:55.872516213 +0000 UTC m=+0.399129464 container start 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T14:52:56.115 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 bash[28311]: 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e 2024-04-25T14:52:56.115 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:55 smithi019 systemd[1]: Started Ceph mgr.x for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:52:56.370 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:56 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:56.113+0000 7f631fff7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:52:56.700 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:56 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:56.380+0000 7f631fff7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:52:56.700 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:56 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:56.517+0000 7f631fff7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:52:57.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:52:56 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/60423147' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:52:57.064 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:56 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:56.946+0000 7f631fff7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:52:57.064 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:57 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:57.062+0000 7f631fff7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:52:57.749 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:52:57.752 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:52:57.753 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:52:57.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:52:38.626431+0000", 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:52:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.698+0000 7f91e1800700 1 Processor -- start 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.698+0000 7f91e1800700 1 -- start start 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.698+0000 7f91e1800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.699+0000 7f91e1800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc063d60 con 0x7f91dc109040 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.699+0000 7f91daa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.699+0000 7f91daa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41754/0 (socket says 172.21.15.19:41754) 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.700+0000 7f91daa00700 1 -- 172.21.15.19:0/3584403520 learned_addr learned my addr 172.21.15.19:0/3584403520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.700+0000 7f91daa00700 1 -- 172.21.15.19:0/3584403520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc10b970 con 0x7f91dc109040 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.700+0000 7f91daa00700 1 --2- 172.21.15.19:0/3584403520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098a0 tx=0x7f91cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93b43dd7edf556e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.701+0000 7f91d9600700 1 -- 172.21.15.19:0/3584403520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc004160 con 0x7f91dc109040 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.701+0000 7f91d9600700 1 -- 172.21.15.19:0/3584403520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f91cc004300 con 0x7f91dc109040 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.701+0000 7f91d9600700 1 -- 172.21.15.19:0/3584403520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc011af0 con 0x7f91dc109040 2024-04-25T14:52:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.701+0000 7f91e1800700 1 -- 172.21.15.19:0/3584403520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 msgr2=0x7f91dc10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.701+0000 7f91e1800700 1 --2- 172.21.15.19:0/3584403520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098a0 tx=0x7f91cc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.702+0000 7f91e1800700 1 -- 172.21.15.19:0/3584403520 shutdown_connections 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.702+0000 7f91e1800700 1 --2- 172.21.15.19:0/3584403520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc10b430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.702+0000 7f91e1800700 1 -- 172.21.15.19:0/3584403520 >> 172.21.15.19:0/3584403520 conn(0x7f91dc102ab0 msgr2=0x7f91dc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.702+0000 7f91e1800700 1 -- 172.21.15.19:0/3584403520 shutdown_connections 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.702+0000 7f91e1800700 1 -- 172.21.15.19:0/3584403520 wait complete. 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.703+0000 7f91e1800700 1 Processor -- start 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.703+0000 7f91e1800700 1 -- start start 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91e1800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91e1800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10c8d0 con 0x7f91dc109040 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91daa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91daa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41770/0 (socket says 172.21.15.19:41770) 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91daa00700 1 -- 172.21.15.19:0/60423147 learned_addr learned my addr 172.21.15.19:0/60423147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.704+0000 7f91daa00700 1 -- 172.21.15.19:0/60423147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc19ebc0 con 0x7f91dc109040 2024-04-25T14:52:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91daa00700 1 --2- 172.21.15.19:0/60423147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f91cc011ac0 tx=0x7f91cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc003b20 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc19ee50 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f91cc01a8c0 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc0248d0 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.705+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1a1b40 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.706+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91cc022020 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.706+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91cc02dde0 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.706+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc0640a0 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.709+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f91cc042e50 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.789+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f91dc1a1fe0 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.790+0000 7f91d3400700 1 -- 172.21.15.19:0/60423147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f91cc031020 con 0x7f91dc109040 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 msgr2=0x7f91dc19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 --2- 172.21.15.19:0/60423147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f91cc011ac0 tx=0x7f91cc011ba0 comp rx=0 tx=0).stop 2024-04-25T14:52:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 shutdown_connections 2024-04-25T14:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 --2- 172.21.15.19:0/60423147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91dc109040 0x7f91dc19e680 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 >> 172.21.15.19:0/60423147 conn(0x7f91dc102ab0 msgr2=0x7f91dc103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 shutdown_connections 2024-04-25T14:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:56.792+0000 7f91e1800700 1 -- 172.21.15.19:0/60423147 wait complete. 2024-04-25T14:52:57.767 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-04-25T14:52:59.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:58.739+0000 7f631fff7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:52:59.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:58.973+0000 7f631fff7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:52:59.586 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:59.191+0000 7f631fff7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:52:59.586 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:59.584+0000 7f631fff7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:52:59.871 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:59.709+0000 7f631fff7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:53:00.213 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:52:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:52:59.969+0000 7f631fff7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:53:00.463 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:00.232+0000 7f631fff7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:00.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1454955244' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:53:01.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:01.104+0000 7f631fff7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:53:01.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:01.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:53:01.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:53:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:53:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:52:38.626431+0000", 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.365+0000 7fc0d5200700 1 Processor -- start 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.365+0000 7fc0d5200700 1 -- start start 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0d5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0d5200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0063ce0 con 0x7fc0d01090c0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0cde00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0cde00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41786/0 (socket says 172.21.15.19:41786) 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0cde00700 1 -- 172.21.15.19:0/2536017239 learned_addr learned my addr 172.21.15.19:0/2536017239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.366+0000 7fc0cde00700 1 -- 172.21.15.19:0/2536017239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d010b9f0 con 0x7fc0d01090c0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.367+0000 7fc0cde00700 1 --2- 172.21.15.19:0/2536017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00098a0 tx=0x7fc0c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ba683c4d3a7272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.367+0000 7fc0cca00700 1 -- 172.21.15.19:0/2536017239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c0004160 con 0x7fc0d01090c0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.367+0000 7fc0cca00700 1 -- 172.21.15.19:0/2536017239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc0c0004300 con 0x7fc0d01090c0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.367+0000 7fc0cca00700 1 -- 172.21.15.19:0/2536017239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c0011af0 con 0x7fc0d01090c0 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 -- 172.21.15.19:0/2536017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 msgr2=0x7fc0d010b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 --2- 172.21.15.19:0/2536017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00098a0 tx=0x7fc0c0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 -- 172.21.15.19:0/2536017239 shutdown_connections 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 --2- 172.21.15.19:0/2536017239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d010b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 -- 172.21.15.19:0/2536017239 >> 172.21.15.19:0/2536017239 conn(0x7fc0d0102ab0 msgr2=0x7fc0d0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:01.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 -- 172.21.15.19:0/2536017239 shutdown_connections 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.368+0000 7fc0d5200700 1 -- 172.21.15.19:0/2536017239 wait complete. 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.369+0000 7fc0d5200700 1 Processor -- start 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.369+0000 7fc0d5200700 1 -- start start 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0d5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0d5200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d010c8e0 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0cde00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0cde00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41802/0 (socket says 172.21.15.19:41802) 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0cde00700 1 -- 172.21.15.19:0/1454955244 learned_addr learned my addr 172.21.15.19:0/1454955244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.370+0000 7fc0cde00700 1 -- 172.21.15.19:0/1454955244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d0108e80 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0cde00700 1 --2- 172.21.15.19:0/1454955244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0011ac0 tx=0x7fc0c0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c0003b20 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc0c001a8c0 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d0107070 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c00248d0 con 0x7fc0d01090c0 2024-04-25T14:53:01.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.371+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d01075b0 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.372+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc0c0022020 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.372+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc0c00193c0 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.372+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d00640a0 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.375+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc0c0042e50 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.454+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc0d0107bd0 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.455+0000 7fc0c6a00700 1 -- 172.21.15.19:0/1454955244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc0c0031020 con 0x7fc0d01090c0 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 msgr2=0x7fc0d0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 --2- 172.21.15.19:0/1454955244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0011ac0 tx=0x7fc0c0011ba0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 shutdown_connections 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 --2- 172.21.15.19:0/1454955244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d01090c0 0x7fc0d0108940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 >> 172.21.15.19:0/1454955244 conn(0x7fc0d0102ab0 msgr2=0x7fc0d0103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 shutdown_connections 2024-04-25T14:53:01.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:00.456+0000 7fc0d5200700 1 -- 172.21.15.19:0/1454955244 wait complete. 2024-04-25T14:53:01.181 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-04-25T14:53:02.074 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:01.673+0000 7f631fff7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:53:02.074 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:01.802+0000 7f631fff7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:53:02.074 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:02.073+0000 7f631fff7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:53:03.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:02.735+0000 7f631fff7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:53:03.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:02.861+0000 7f631fff7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:53:03.461 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:03.190+0000 7f631fff7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:53:03.462 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:03.380+0000 7f631fff7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:53:03.775 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:03.512+0000 7f631fff7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:53:03.775 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:03.669+0000 7f631fff7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:04.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:03 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2389791803' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:53:04.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:04 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:04.055+0000 7f631fff7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:53:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:53:04.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:53:04.722 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.723 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:52:38.626431+0000", 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:04.724 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.819+0000 7f0f1ea00700 1 Processor -- start 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.819+0000 7f0f1ea00700 1 -- start start 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.820+0000 7f0f1ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.820+0000 7f0f1ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18108fb0 con 0x7f0f181085c0 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.820+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.820+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41808/0 (socket says 172.21.15.19:41808) 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.820+0000 7f0f17e00700 1 -- 172.21.15.19:0/1210891533 learned_addr learned my addr 172.21.15.19:0/1210891533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.821+0000 7f0f17e00700 1 -- 172.21.15.19:0/1210891533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181097c0 con 0x7f0f181085c0 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.821+0000 7f0f17e00700 1 --2- 172.21.15.19:0/1210891533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995b937230ee9f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.821+0000 7f0f16a00700 1 -- 172.21.15.19:0/1210891533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08004160 con 0x7f0f181085c0 2024-04-25T14:53:04.725 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.821+0000 7f0f16a00700 1 -- 172.21.15.19:0/1210891533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f08004300 con 0x7f0f181085c0 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.821+0000 7f0f16a00700 1 -- 172.21.15.19:0/1210891533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08011af0 con 0x7f0f181085c0 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 -- 172.21.15.19:0/1210891533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 msgr2=0x7f0f181089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 --2- 172.21.15.19:0/1210891533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 -- 172.21.15.19:0/1210891533 shutdown_connections 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 --2- 172.21.15.19:0/1210891533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f181089e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 -- 172.21.15.19:0/1210891533 >> 172.21.15.19:0/1210891533 conn(0x7f0f18103be0 msgr2=0x7f0f18106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 -- 172.21.15.19:0/1210891533 shutdown_connections 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.822+0000 7f0f1ea00700 1 -- 172.21.15.19:0/1210891533 wait complete. 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.823+0000 7f0f1ea00700 1 Processor -- start 2024-04-25T14:53:04.726 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f1ea00700 1 -- start start 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f1ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f1ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810c9f0 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41810/0 (socket says 172.21.15.19:41810) 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.824+0000 7f0f17e00700 1 -- 172.21.15.19:0/2389791803 learned_addr learned my addr 172.21.15.19:0/2389791803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f17e00700 1 -- 172.21.15.19:0/2389791803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1819ec60 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f17e00700 1 --2- 172.21.15.19:0/2389791803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117b0 tx=0x7f0f08011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08003af0 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f08020070 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08019630 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1819eef0 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.825+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a1be0 con 0x7f0f181085c0 2024-04-25T14:53:04.727 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.826+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f08019c30 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.826+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f0800b720 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.826+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1810cb70 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.831+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0f0801a9c0 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.911+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0f181a20b0 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.911+0000 7f0f14c00700 1 -- 172.21.15.19:0/2389791803 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f0f0801a430 con 0x7f0f181085c0 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.913+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 msgr2=0x7f0f1819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.913+0000 7f0f1ea00700 1 --2- 172.21.15.19:0/2389791803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117b0 tx=0x7f0f08011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.913+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 shutdown_connections 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.913+0000 7f0f1ea00700 1 --2- 172.21.15.19:0/2389791803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f181085c0 0x7f0f1819e720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.913+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 >> 172.21.15.19:0/2389791803 conn(0x7f0f18103be0 msgr2=0x7f0f18104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.914+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 shutdown_connections 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:03.914+0000 7f0f1ea00700 1 -- 172.21.15.19:0/2389791803 wait complete. 2024-04-25T14:53:04.728 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (3/15)... 2024-04-25T14:53:05.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:04 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:04.849+0000 7f631fff7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:53:05.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:05.164+0000 7f631fff7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:53:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: Activating manager daemon x 2024-04-25T14:53:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: mgrmap e2: x(active, starting, since 0.011521s) 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: Manager daemon x is now available 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' 2024-04-25T14:53:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:05 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:53:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:06 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' 2024-04-25T14:53:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:06 smithi019 ceph-mon[28090]: from='mgr.14100 172.21.15.19:0/3351014823' entity='mgr.x' 2024-04-25T14:53:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:06 smithi019 ceph-mon[28090]: mgrmap e3: x(active, since 1.01954s) 2024-04-25T14:53:08.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:08 smithi019 ceph-mon[28090]: mgrmap e4: x(active, since 2s) 2024-04-25T14:53:08.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:08 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3868144249' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:53:08.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:08.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:08.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T14:53:08.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:53:08.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:53:08.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:53:08.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T14:53:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:53:08.381 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:08.382 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:53:08.382 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:53:08.382 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.382 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:53:08.384 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:53:08.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:53:08.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:52:38.626431+0000", 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.347+0000 7fa257e00700 1 Processor -- start 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.348+0000 7fa257e00700 1 -- start start 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.348+0000 7fa257e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.348+0000 7fa257e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa258107d20 con 0x7fa258107330 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.348+0000 7fa256a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.349+0000 7fa256a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33836/0 (socket says 172.21.15.19:33836) 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.349+0000 7fa256a00700 1 -- 172.21.15.19:0/3457457608 learned_addr learned my addr 172.21.15.19:0/3457457608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.349+0000 7fa256a00700 1 -- 172.21.15.19:0/3457457608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa258108530 con 0x7fa258107330 2024-04-25T14:53:08.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.349+0000 7fa256a00700 1 --2- 172.21.15.19:0/3457457608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2480098a0 tx=0x7fa248009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5753ffa13a4c74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa255600700 1 -- 172.21.15.19:0/3457457608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa248004160 con 0x7fa258107330 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa255600700 1 -- 172.21.15.19:0/3457457608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa248004300 con 0x7fa258107330 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa255600700 1 -- 172.21.15.19:0/3457457608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa248011af0 con 0x7fa258107330 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa257e00700 1 -- 172.21.15.19:0/3457457608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 msgr2=0x7fa258107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa257e00700 1 --2- 172.21.15.19:0/3457457608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2480098a0 tx=0x7fa248009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa257e00700 1 -- 172.21.15.19:0/3457457608 shutdown_connections 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa257e00700 1 --2- 172.21.15.19:0/3457457608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa258107750 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.350+0000 7fa257e00700 1 -- 172.21.15.19:0/3457457608 >> 172.21.15.19:0/3457457608 conn(0x7fa2581028d0 msgr2=0x7fa258104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.351+0000 7fa257e00700 1 -- 172.21.15.19:0/3457457608 shutdown_connections 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.351+0000 7fa257e00700 1 -- 172.21.15.19:0/3457457608 wait complete. 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa257e00700 1 Processor -- start 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa257e00700 1 -- start start 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa257e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:08.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa257e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810b760 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa256a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa256a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33838/0 (socket says 172.21.15.19:33838) 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.352+0000 7fa256a00700 1 -- 172.21.15.19:0/3868144249 learned_addr learned my addr 172.21.15.19:0/3868144249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.353+0000 7fa256a00700 1 -- 172.21.15.19:0/3868144249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819eb60 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.353+0000 7fa256a00700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa248000c00 tx=0x7fa248011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.353+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa248003b20 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.353+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa24801a8c0 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.353+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25819edf0 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.354+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa248024940 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.354+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a1a90 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.355+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25810b8e0 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.355+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa248022020 con 0x7fa258107330 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.356+0000 7fa24f400700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 0x7fa23c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:08.397 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.356+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa24804d420 con 0x7fa258107330 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.356+0000 7fa256000700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 0x7fa23c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.357+0000 7fa256000700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 0x7fa23c03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa244006fd0 tx=0x7fa244006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.362+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa24802c3a0 con 0x7fa258107330 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.684+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa25802e180 con 0x7fa258107330 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.685+0000 7fa24f400700 1 -- 172.21.15.19:0/3868144249 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa24800be10 con 0x7fa258107330 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 msgr2=0x7fa23c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 0x7fa23c03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa244006fd0 tx=0x7fa244006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 msgr2=0x7fa25819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa248000c00 tx=0x7fa248011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 shutdown_connections 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fa23c0395b0 0x7fa23c03ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 --2- 172.21.15.19:0/3868144249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa258107330 0x7fa25819e620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 >> 172.21.15.19:0/3868144249 conn(0x7fa2581028d0 msgr2=0x7fa258103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:08.398 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 shutdown_connections 2024-04-25T14:53:08.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:07.690+0000 7fa257e00700 1 -- 172.21.15.19:0/3868144249 wait complete. 2024-04-25T14:53:08.399 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-04-25T14:53:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:09 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1399860946' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:53:10.038 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:53:10.039 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:53:10.042 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.021+0000 7fbce6800700 1 Processor -- start 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.022+0000 7fbce6800700 1 -- start start 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.022+0000 7fbce6800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.022+0000 7fbce6800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010b1f0 con 0x7fbce010a800 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.023+0000 7fbcdf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:10.043 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.023+0000 7fbcdf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33854/0 (socket says 172.21.15.19:33854) 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.023+0000 7fbcdf400700 1 -- 172.21.15.19:0/2436912413 learned_addr learned my addr 172.21.15.19:0/2436912413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.023+0000 7fbcdf400700 1 -- 172.21.15.19:0/2436912413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce010b9b0 con 0x7fbce010a800 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.024+0000 7fbcdf400700 1 --2- 172.21.15.19:0/2436912413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b391662dcf53d8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.024+0000 7fbcde000700 1 -- 172.21.15.19:0/2436912413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0004160 con 0x7fbce010a800 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.024+0000 7fbcde000700 1 -- 172.21.15.19:0/2436912413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbcd0004300 con 0x7fbce010a800 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.024+0000 7fbcde000700 1 -- 172.21.15.19:0/2436912413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0011af0 con 0x7fbce010a800 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 -- 172.21.15.19:0/2436912413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 msgr2=0x7fbce010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 --2- 172.21.15.19:0/2436912413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 -- 172.21.15.19:0/2436912413 shutdown_connections 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 --2- 172.21.15.19:0/2436912413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce010ac20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 -- 172.21.15.19:0/2436912413 >> 172.21.15.19:0/2436912413 conn(0x7fbce007ddf0 msgr2=0x7fbce01084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 -- 172.21.15.19:0/2436912413 shutdown_connections 2024-04-25T14:53:10.044 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.025+0000 7fbce6800700 1 -- 172.21.15.19:0/2436912413 wait complete. 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.026+0000 7fbce6800700 1 Processor -- start 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.026+0000 7fbce6800700 1 -- start start 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.027+0000 7fbce6800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.027+0000 7fbce6800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010ebe0 con 0x7fbce010a800 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.027+0000 7fbcdf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.027+0000 7fbcdf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33856/0 (socket says 172.21.15.19:33856) 2024-04-25T14:53:10.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.027+0000 7fbcdf400700 1 -- 172.21.15.19:0/1399860946 learned_addr learned my addr 172.21.15.19:0/1399860946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbcdf400700 1 -- 172.21.15.19:0/1399860946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce019ebc0 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbcdf400700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0000c00 tx=0x7fbcd0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0003b20 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbcd001a8c0 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0024940 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce019ee50 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.028+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a1b40 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.030+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fbcd0022020 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.030+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce010ed60 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.030+0000 7fbcd7e00700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 0x7fbcc403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.030+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbcd004d450 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.030+0000 7fbcdea00700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 0x7fbcc403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.031+0000 7fbcdea00700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 0x7fbcc403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.038+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcd000be50 con 0x7fbce010a800 2024-04-25T14:53:10.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.267+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbce00640a0 con 0x7fbce010a800 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.269+0000 7fbcd7e00700 1 -- 172.21.15.19:0/1399860946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fbcd004d1b0 con 0x7fbce010a800 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.272+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 msgr2=0x7fbcc403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.272+0000 7fbce6800700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 0x7fbcc403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.272+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 msgr2=0x7fbce019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.272+0000 7fbce6800700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0000c00 tx=0x7fbcd0011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 shutdown_connections 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fbcc40395b0 0x7fbcc403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 --2- 172.21.15.19:0/1399860946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce010a800 0x7fbce019e680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 >> 172.21.15.19:0/1399860946 conn(0x7fbce007ddf0 msgr2=0x7fbce0107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 shutdown_connections 2024-04-25T14:53:10.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:09.273+0000 7fbce6800700 1 -- 172.21.15.19:0/1399860946 wait complete. 2024-04-25T14:53:10.051 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-04-25T14:53:11.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2253782742' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T14:53:11.363 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setuser ceph since I am not root 2024-04-25T14:53:11.363 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setgroup ceph since I am not root 2024-04-25T14:53:11.363 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:11.361+0000 7f8de2e2b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:53:11.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:11.605+0000 7f8de2e2b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.713+0000 7f973d400700 1 Processor -- start 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.713+0000 7f973d400700 1 -- start start 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.714+0000 7f973d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.714+0000 7f973d400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9738107ec0 con 0x7f97381074d0 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.714+0000 7f9736800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:11.758 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.714+0000 7f9736800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33860/0 (socket says 172.21.15.19:33860) 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.714+0000 7f9736800700 1 -- 172.21.15.19:0/809395503 learned_addr learned my addr 172.21.15.19:0/809395503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.715+0000 7f9736800700 1 -- 172.21.15.19:0/809395503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97381086d0 con 0x7f97381074d0 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.715+0000 7f9736800700 1 --2- 172.21.15.19:0/809395503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f972800bb40 tx=0x7f972800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e31af95473d1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.715+0000 7f9735400700 1 -- 172.21.15.19:0/809395503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f972800dbe0 con 0x7f97381074d0 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f9735400700 1 -- 172.21.15.19:0/809395503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f97280044c0 con 0x7f97381074d0 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f9735400700 1 -- 172.21.15.19:0/809395503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9728010470 con 0x7f97381074d0 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 -- 172.21.15.19:0/809395503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 msgr2=0x7f97381078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 --2- 172.21.15.19:0/809395503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f972800bb40 tx=0x7f972800be50 comp rx=0 tx=0).stop 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 -- 172.21.15.19:0/809395503 shutdown_connections 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 --2- 172.21.15.19:0/809395503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f97381078f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 -- 172.21.15.19:0/809395503 >> 172.21.15.19:0/809395503 conn(0x7f9738102ab0 msgr2=0x7f9738104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 -- 172.21.15.19:0/809395503 shutdown_connections 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.716+0000 7f973d400700 1 -- 172.21.15.19:0/809395503 wait complete. 2024-04-25T14:53:11.759 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.717+0000 7f973d400700 1 Processor -- start 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f973d400700 1 -- start start 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f973d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f973d400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9738062e80 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f9736800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f9736800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33872/0 (socket says 172.21.15.19:33872) 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f9736800700 1 -- 172.21.15.19:0/2253782742 learned_addr learned my addr 172.21.15.19:0/2253782742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.718+0000 7f9736800700 1 -- 172.21.15.19:0/2253782742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973819eba0 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f9736800700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f972800bfd0 tx=0x7f9728003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9728010770 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f972801d070 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9728019c70 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973819ee30 con 0x7f97381074d0 2024-04-25T14:53:11.760 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.719+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97381a1b20 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.720+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f9728028060 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.720+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97380640a0 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.721+0000 7f972f400700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 0x7f971c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.721+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f972804d900 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.721+0000 7f9735e00700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 0x7f971c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.722+0000 7f9735e00700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 0x7f971c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9724006fd0 tx=0x7f9724006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:10.726+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9728015070 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.020+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f97381a1cf0 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.074+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f97381a1cf0 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.076+0000 7f972f400700 1 -- 172.21.15.19:0/2253782742 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f972801f030 con 0x7f97381074d0 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 msgr2=0x7f971c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 0x7f971c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9724006fd0 tx=0x7f9724006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 msgr2=0x7f973819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:11.761 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f972800bfd0 tx=0x7f9728003ff0 comp rx=0 tx=0).stop 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 shutdown_connections 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f971c039560 0x7f971c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 --2- 172.21.15.19:0/2253782742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97381074d0 0x7f973819e660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 >> 172.21.15.19:0/2253782742 conn(0x7f9738102ab0 msgr2=0x7f9738194ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.078+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 shutdown_connections 2024-04-25T14:53:11.762 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:11.079+0000 7f973d400700 1 -- 172.21.15.19:0/2253782742 wait complete. 2024-04-25T14:53:12.023 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:11.741+0000 7f8de2e2b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:53:12.299 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:12.183+0000 7f8de2e2b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:53:12.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:12 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2253782742' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T14:53:12.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:12 smithi019 ceph-mon[28090]: mgrmap e5: x(active, since 5s) 2024-04-25T14:53:12.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:12.298+0000 7f8de2e2b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:53:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:13 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2516992372' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:53:13.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:13.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T14:53:13.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:53:13.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:53:13.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:53:13.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:13.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.474+0000 7fdc9ea00700 1 Processor -- start 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.475+0000 7fdc9ea00700 1 -- start start 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.475+0000 7fdc9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.475+0000 7fdc9ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9807b7c0 con 0x7fdc9807cd90 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.476+0000 7fdc97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.476+0000 7fdc97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33888/0 (socket says 172.21.15.19:33888) 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.476+0000 7fdc97e00700 1 -- 172.21.15.19:0/2961242491 learned_addr learned my addr 172.21.15.19:0/2961242491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.476+0000 7fdc97e00700 1 -- 172.21.15.19:0/2961242491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9807b940 con 0x7fdc9807cd90 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.476+0000 7fdc97e00700 1 --2- 172.21.15.19:0/2961242491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098a0 tx=0x7fdc88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48142400243fb888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.477+0000 7fdc96a00700 1 -- 172.21.15.19:0/2961242491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88004160 con 0x7fdc9807cd90 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.477+0000 7fdc96a00700 1 -- 172.21.15.19:0/2961242491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdc88004300 con 0x7fdc9807cd90 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.477+0000 7fdc96a00700 1 -- 172.21.15.19:0/2961242491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88011af0 con 0x7fdc9807cd90 2024-04-25T14:53:13.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.477+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2961242491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 msgr2=0x7fdc9807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.477+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2961242491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098a0 tx=0x7fdc88009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.478+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2961242491 shutdown_connections 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.478+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2961242491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9807b1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.478+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2961242491 >> 172.21.15.19:0/2961242491 conn(0x7fdc98103be0 msgr2=0x7fdc98106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.478+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2961242491 shutdown_connections 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.478+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2961242491 wait complete. 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.479+0000 7fdc9ea00700 1 Processor -- start 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.479+0000 7fdc9ea00700 1 -- start start 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.479+0000 7fdc9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.479+0000 7fdc9ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9810eaf0 con 0x7fdc9807cd90 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.480+0000 7fdc97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.480+0000 7fdc97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33902/0 (socket says 172.21.15.19:33902) 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.480+0000 7fdc97e00700 1 -- 172.21.15.19:0/2516992372 learned_addr learned my addr 172.21.15.19:0/2516992372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.480+0000 7fdc97e00700 1 -- 172.21.15.19:0/2516992372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9819ec50 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.480+0000 7fdc97e00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc880117b0 tx=0x7fdc88011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.481+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88003af0 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.481+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdc88020070 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.481+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc88019630 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.481+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9819eee0 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.481+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a1bd0 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.482+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc980640a0 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.482+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fdc8801a9c0 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.483+0000 7fdc94c00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 0x7fdc7c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.483+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc8804db20 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.483+0000 7fdc97400700 1 -- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 msgr2=0x7fdc7c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.483+0000 7fdc97400700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 0x7fdc7c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.488+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc8800e260 con 0x7fdc9807cd90 2024-04-25T14:53:13.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.684+0000 7fdc97400700 1 -- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 msgr2=0x7fdc7c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.684+0000 7fdc97400700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 0x7fdc7c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.778+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdc981a1da0 con 0x7fdc9807cd90 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.779+0000 7fdc94c00700 1 -- 172.21.15.19:0/2516992372 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fdc8804d540 con 0x7fdc9807cd90 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.781+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 msgr2=0x7fdc7c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.781+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 0x7fdc7c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.781+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 msgr2=0x7fdc9819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc880117b0 tx=0x7fdc88011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 shutdown_connections 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7fdc7c039600 0x7fdc7c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 --2- 172.21.15.19:0/2516992372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc9807cd90 0x7fdc9819e710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 >> 172.21.15.19:0/2516992372 conn(0x7fdc98103be0 msgr2=0x7fdc98104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 shutdown_connections 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:12.782+0000 7fdc9ea00700 1 -- 172.21.15.19:0/2516992372 wait complete. 2024-04-25T14:53:13.529 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T14:53:13.530 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-04-25T14:53:14.120 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:13.906+0000 7f8de2e2b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:53:14.403 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:14.142+0000 7f8de2e2b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:14.403 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:14.361+0000 7f8de2e2b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:53:15.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:14.754+0000 7f8de2e2b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:53:15.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:14.879+0000 7f8de2e2b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:53:15.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:15.137+0000 7f8de2e2b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:53:15.398 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:15.397+0000 7f8de2e2b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:16.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:16.209+0000 7f8de2e2b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:53:17.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:16.778+0000 7f8de2e2b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:53:17.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:16.908+0000 7f8de2e2b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:53:17.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:17.182+0000 7f8de2e2b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:53:18.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:17.849+0000 7f8de2e2b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:53:18.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:17.977+0000 7f8de2e2b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:53:18.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:18.309+0000 7f8de2e2b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:53:18.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:18.493+0000 7f8de2e2b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:53:19.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:18.622+0000 7f8de2e2b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:53:19.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:18.777+0000 7f8de2e2b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:19.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:19.151+0000 7f8de2e2b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:53:20.226 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:19.932+0000 7f8de2e2b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:20.225+0000 7f8de2e2b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: Active manager daemon x restarted 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: Activating manager daemon x 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: mgrmap e6: x(active, starting, since 0.0104172s) 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:53:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:53:20.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:53:20.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:53:20.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: Manager daemon x is now available 2024-04-25T14:53:20.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:20 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:21.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: Found migration_current of "None". Setting to last migration. 2024-04-25T14:53:21.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:21.615 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:21.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:21.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:53:21.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:53:21.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:21 smithi019 ceph-mon[28090]: mgrmap e7: x(active, since 1.01705s) 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.193+0000 7f5502000700 1 Processor -- start 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f5502000700 1 -- start start 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f5502000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f5502000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc07b780 con 0x7f54fc07cd50 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f54fb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:21.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f54fb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55718/0 (socket says 172.21.15.19:55718) 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.194+0000 7f54fb400700 1 -- 172.21.15.19:0/305602274 learned_addr learned my addr 172.21.15.19:0/305602274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.195+0000 7f54fb400700 1 -- 172.21.15.19:0/305602274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc07b900 con 0x7f54fc07cd50 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.195+0000 7f54fb400700 1 --2- 172.21.15.19:0/305602274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0098a0 tx=0x7f54ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599e5b0df05fde92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.195+0000 7f54fa000700 1 -- 172.21.15.19:0/305602274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54ec004160 con 0x7f54fc07cd50 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.195+0000 7f54fa000700 1 -- 172.21.15.19:0/305602274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54ec004300 con 0x7f54fc07cd50 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.195+0000 7f54fa000700 1 -- 172.21.15.19:0/305602274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54ec011af0 con 0x7f54fc07cd50 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.196+0000 7f5502000700 1 -- 172.21.15.19:0/305602274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 msgr2=0x7f54fc07b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.196+0000 7f5502000700 1 --2- 172.21.15.19:0/305602274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0098a0 tx=0x7f54ec009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.197+0000 7f5502000700 1 -- 172.21.15.19:0/305602274 shutdown_connections 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.197+0000 7f5502000700 1 --2- 172.21.15.19:0/305602274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc07b1b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.197+0000 7f5502000700 1 -- 172.21.15.19:0/305602274 >> 172.21.15.19:0/305602274 conn(0x7f54fc103be0 msgr2=0x7f54fc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.197+0000 7f5502000700 1 -- 172.21.15.19:0/305602274 shutdown_connections 2024-04-25T14:53:21.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.197+0000 7f5502000700 1 -- 172.21.15.19:0/305602274 wait complete. 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.198+0000 7f5502000700 1 Processor -- start 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.198+0000 7f5502000700 1 -- start start 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.198+0000 7f5502000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.198+0000 7f5502000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10eb30 con 0x7f54fc07cd50 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54fb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54fb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55734/0 (socket says 172.21.15.19:55734) 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54fb400700 1 -- 172.21.15.19:0/1188568602 learned_addr learned my addr 172.21.15.19:0/1188568602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54fb400700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc19ec20 con 0x7f54fc07cd50 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54fb400700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009fb0 tx=0x7f54ec011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.199+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54ec0036a0 con 0x7f54fc07cd50 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.200+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54ec01a430 con 0x7f54fc07cd50 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.200+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc19eeb0 con 0x7f54fc07cd50 2024-04-25T14:53:21.946 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.200+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54ec019550 con 0x7f54fc07cd50 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.200+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1a1ba0 con 0x7f54fc07cd50 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f54ec019850 con 0x7f54fc07cd50 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f54f3e00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f54fc069b80 con 0x7f54e0039560 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54ec04d930 con 0x7f54fc07cd50 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f54faa00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.201+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.402+0000 7f54faa00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.402+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.803+0000 7f54faa00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:14.803+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:15.604+0000 7f54faa00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:15.604+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:53:21.947 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:17.206+0000 7f54faa00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2453471633 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:17.206+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:20.236+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f54ec02d960 con 0x7f54fc07cd50 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:20.236+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 msgr2=0x7f54e003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:20.236+0000 7f54f3e00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.240+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f54ec018350 con 0x7f54fc07cd50 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.241+0000 7f54f3e00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 0x7f54e003f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.241+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f54ec001d30 con 0x7f54e003cc60 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.243+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 0x7f54e003f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.247+0000 7f54faa00700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 0x7f54e003f050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f54e8003a10 tx=0x7f54e8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.249+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f54ec001d30 con 0x7f54e003cc60 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.262+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f54fc19f390 con 0x7f54e003cc60 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.262+0000 7f54f3e00700 1 -- 172.21.15.19:0/1188568602 <== mgr.14120 v2:172.21.15.19:6800/2740045356 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f54fc19f390 con 0x7f54e003cc60 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 msgr2=0x7f54e003f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:21.948 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 0x7f54e003f050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f54e8003a10 tx=0x7f54e8009290 comp rx=0 tx=0).stop 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 msgr2=0x7f54fc19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009fb0 tx=0x7f54ec011940 comp rx=0 tx=0).stop 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 shutdown_connections 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f54e003cc60 0x7f54e003f050 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54fc07cd50 0x7f54fc19e6e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 --2- 172.21.15.19:0/1188568602 >> [v2:172.21.15.19:6800/2453471633,v1:172.21.15.19:6801/2453471633] conn(0x7f54e0039560 0x7f54e003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.263+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 >> 172.21.15.19:0/1188568602 conn(0x7f54fc103be0 msgr2=0x7f54fc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.264+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 shutdown_connections 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:21.264+0000 7f5502000700 1 -- 172.21.15.19:0/1188568602 wait complete. 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-04-25T14:53:21.949 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-04-25T14:53:22.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:22 smithi019 ceph-mon[28090]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:53:22.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:22 smithi019 ceph-mon[28090]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:53:22.504 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:22 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:22.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:22 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: mgrmap e8: x(active, since 2s) 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:22] ENGINE Bus STARTING 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:22] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:22] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:22] ENGINE Bus STARTED 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:23.533 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:23.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:23.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:23 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:23.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.503+0000 7faf4e800700 1 Processor -- start 2024-04-25T14:53:23.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.503+0000 7faf4e800700 1 -- start start 2024-04-25T14:53:23.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.504+0000 7faf4e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:23.545 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.504+0000 7faf4e800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf4807b780 con 0x7faf4807cd50 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.504+0000 7faf47400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.504+0000 7faf47400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55800/0 (socket says 172.21.15.19:55800) 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.504+0000 7faf47400700 1 -- 172.21.15.19:0/2890607075 learned_addr learned my addr 172.21.15.19:0/2890607075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.505+0000 7faf47400700 1 -- 172.21.15.19:0/2890607075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4807b900 con 0x7faf4807cd50 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.505+0000 7faf47400700 1 --2- 172.21.15.19:0/2890607075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf380098a0 tx=0x7faf38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba30a45c362da46a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf46000700 1 -- 172.21.15.19:0/2890607075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf38004160 con 0x7faf4807cd50 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf46000700 1 -- 172.21.15.19:0/2890607075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf38004300 con 0x7faf4807cd50 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf46000700 1 -- 172.21.15.19:0/2890607075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf38011af0 con 0x7faf4807cd50 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 -- 172.21.15.19:0/2890607075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 msgr2=0x7faf4807b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 --2- 172.21.15.19:0/2890607075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf380098a0 tx=0x7faf38009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:23.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 -- 172.21.15.19:0/2890607075 shutdown_connections 2024-04-25T14:53:23.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 --2- 172.21.15.19:0/2890607075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf4807b1b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:23.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 -- 172.21.15.19:0/2890607075 >> 172.21.15.19:0/2890607075 conn(0x7faf48103be0 msgr2=0x7faf48106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:23.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 -- 172.21.15.19:0/2890607075 shutdown_connections 2024-04-25T14:53:23.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.506+0000 7faf4e800700 1 -- 172.21.15.19:0/2890607075 wait complete. 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.508+0000 7faf4e800700 1 Processor -- start 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.508+0000 7faf4e800700 1 -- start start 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.508+0000 7faf4e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.508+0000 7faf4e800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf4810a720 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.509+0000 7faf47400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.509+0000 7faf47400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55812/0 (socket says 172.21.15.19:55812) 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.509+0000 7faf47400700 1 -- 172.21.15.19:0/3626164345 learned_addr learned my addr 172.21.15.19:0/3626164345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.509+0000 7faf47400700 1 -- 172.21.15.19:0/3626164345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf481a3010 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.509+0000 7faf47400700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faf38011ac0 tx=0x7faf38011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.510+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf38003970 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.510+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faf38003b10 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.510+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf38024610 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.510+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf481a32a0 con 0x7faf4807cd50 2024-04-25T14:53:23.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.510+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf481a5f90 con 0x7faf4807cd50 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.511+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf480640a0 con 0x7faf4807cd50 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.511+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faf3801a9e0 con 0x7faf4807cd50 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.512+0000 7faf3fe00700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 0x7faf2c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.512+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf3804d8f0 con 0x7faf4807cd50 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.512+0000 7faf46a00700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 0x7faf2c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.513+0000 7faf46a00700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 0x7faf2c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.518+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf3800b3e0 con 0x7faf4807cd50 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.766+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7faf4802e1c0 con 0x7faf2c039600 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.779+0000 7faf3fe00700 1 -- 172.21.15.19:0/3626164345 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf4802e1c0 con 0x7faf2c039600 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 msgr2=0x7faf2c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 0x7faf2c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 msgr2=0x7faf481a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faf38011ac0 tx=0x7faf38011ba0 comp rx=0 tx=0).stop 2024-04-25T14:53:23.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 shutdown_connections 2024-04-25T14:53:23.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7faf2c039600 0x7faf2c03bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:23.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 --2- 172.21.15.19:0/3626164345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4807cd50 0x7faf481a2ad0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:23.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.782+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 >> 172.21.15.19:0/3626164345 conn(0x7faf48103be0 msgr2=0x7faf48105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:23.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.783+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 shutdown_connections 2024-04-25T14:53:23.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:22.783+0000 7faf4e800700 1 -- 172.21.15.19:0/3626164345 wait complete. 2024-04-25T14:53:25.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T14:53:25.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.249+0000 7efc85e00700 1 Processor -- start 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.249+0000 7efc85e00700 1 -- start start 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.249+0000 7efc85e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.249+0000 7efc85e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc80108fd0 con 0x7efc801085e0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.250+0000 7efc7ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.250+0000 7efc7ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36100/0 (socket says 172.21.15.19:36100) 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.250+0000 7efc7ea00700 1 -- 172.21.15.19:0/2184245692 learned_addr learned my addr 172.21.15.19:0/2184245692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.250+0000 7efc7ea00700 1 -- 172.21.15.19:0/2184245692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc80109790 con 0x7efc801085e0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.251+0000 7efc7ea00700 1 --2- 172.21.15.19:0/2184245692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc700098a0 tx=0x7efc70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3a36267e8c166e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.251+0000 7efc7d600700 1 -- 172.21.15.19:0/2184245692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70004160 con 0x7efc801085e0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.251+0000 7efc7d600700 1 -- 172.21.15.19:0/2184245692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efc70004300 con 0x7efc801085e0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.251+0000 7efc7d600700 1 -- 172.21.15.19:0/2184245692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70011af0 con 0x7efc801085e0 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 -- 172.21.15.19:0/2184245692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 msgr2=0x7efc80108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 --2- 172.21.15.19:0/2184245692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc700098a0 tx=0x7efc70009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 -- 172.21.15.19:0/2184245692 shutdown_connections 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 --2- 172.21.15.19:0/2184245692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc80108a00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 -- 172.21.15.19:0/2184245692 >> 172.21.15.19:0/2184245692 conn(0x7efc80103c60 msgr2=0x7efc80106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 -- 172.21.15.19:0/2184245692 shutdown_connections 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.252+0000 7efc85e00700 1 -- 172.21.15.19:0/2184245692 wait complete. 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.253+0000 7efc85e00700 1 Processor -- start 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.253+0000 7efc85e00700 1 -- start start 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc85e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc85e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010c9c0 con 0x7efc801085e0 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc7ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc7ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36102/0 (socket says 172.21.15.19:36102) 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc7ea00700 1 -- 172.21.15.19:0/3366258047 learned_addr learned my addr 172.21.15.19:0/3366258047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.254+0000 7efc7ea00700 1 -- 172.21.15.19:0/3366258047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8019ec60 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc7ea00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc700117b0 tx=0x7efc70011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70003af0 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efc70020070 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70019630 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8019eef0 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.255+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a1be0 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.256+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efc7001a9c0 con 0x7efc801085e0 2024-04-25T14:53:25.184 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.257+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8010cb40 con 0x7efc801085e0 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.257+0000 7efc84a00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 0x7efc6403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.257+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc7004dbf0 con 0x7efc801085e0 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.257+0000 7efc7e000700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 0x7efc6403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.258+0000 7efc7e000700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 0x7efc6403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc68006fd0 tx=0x7efc68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.263+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc700042a0 con 0x7efc801085e0 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.497+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7efc80104fb0 con 0x7efc64039660 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.499+0000 7efc84a00700 1 -- 172.21.15.19:0/3366258047 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7efc80104fb0 con 0x7efc64039660 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 msgr2=0x7efc6403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 0x7efc6403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc68006fd0 tx=0x7efc68006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 msgr2=0x7efc8019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc700117b0 tx=0x7efc70011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 shutdown_connections 2024-04-25T14:53:25.185 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7efc64039660 0x7efc6403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.186 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.502+0000 7efc85e00700 1 --2- 172.21.15.19:0/3366258047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc801085e0 0x7efc8019e720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.186 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.503+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 >> 172.21.15.19:0/3366258047 conn(0x7efc80103c60 msgr2=0x7efc801048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:25.186 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.503+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 shutdown_connections 2024-04-25T14:53:25.186 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:24.503+0000 7efc85e00700 1 -- 172.21.15.19:0/3366258047 wait complete. 2024-04-25T14:53:25.186 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-04-25T14:53:25.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:25 smithi019 ceph-mon[28090]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:26.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-mon[28090]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:26.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-mon[28090]: Generating ssh key... 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: Generating public/private rsa key pair. 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: Your identification has been saved in /tmp/tmpm5tnxxev/key. 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: Your public key has been saved in /tmp/tmpm5tnxxev/key.pub. 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: The key fingerprint is: 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: SHA256:xjnnS+XtWqOWI5ffSJzmAwkedoo1D9OVsyDk8n097kk ceph-42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: The key's randomart image is: 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: +---[RSA 3072]----+ 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | .. . | 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | .. . + | 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | . .o o o | 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | . +O.o .. | 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | S=oXo....| 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | ..+oo++...| 2024-04-25T14:53:26.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | o .+OE | 2024-04-25T14:53:26.373 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | ...*B++.| 2024-04-25T14:53:26.373 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: | .+o+=+.| 2024-04-25T14:53:26.373 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: +----[SHA256]-----+ 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.833+0000 7f142f200700 1 Processor -- start 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.833+0000 7f142f200700 1 -- start start 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.834+0000 7f142f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.834+0000 7f142f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1430108df0 con 0x7f1430108400 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.834+0000 7f142de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.834+0000 7f142de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36104/0 (socket says 172.21.15.19:36104) 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.834+0000 7f142de00700 1 -- 172.21.15.19:0/335932678 learned_addr learned my addr 172.21.15.19:0/335932678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.835+0000 7f142de00700 1 -- 172.21.15.19:0/335932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1430109600 con 0x7f1430108400 2024-04-25T14:53:27.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.835+0000 7f142de00700 1 --2- 172.21.15.19:0/335932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14200098a0 tx=0x7f1420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dfaceabc265233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.835+0000 7f142ca00700 1 -- 172.21.15.19:0/335932678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420004160 con 0x7f1430108400 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.835+0000 7f142ca00700 1 -- 172.21.15.19:0/335932678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1420004300 con 0x7f1430108400 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.835+0000 7f142ca00700 1 -- 172.21.15.19:0/335932678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420011af0 con 0x7f1430108400 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 -- 172.21.15.19:0/335932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 msgr2=0x7f1430108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 --2- 172.21.15.19:0/335932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14200098a0 tx=0x7f1420009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 -- 172.21.15.19:0/335932678 shutdown_connections 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 --2- 172.21.15.19:0/335932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f1430108820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 -- 172.21.15.19:0/335932678 >> 172.21.15.19:0/335932678 conn(0x7f1430103a00 msgr2=0x7f1430105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 -- 172.21.15.19:0/335932678 shutdown_connections 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.836+0000 7f142f200700 1 -- 172.21.15.19:0/335932678 wait complete. 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.837+0000 7f142f200700 1 Processor -- start 2024-04-25T14:53:27.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.837+0000 7f142f200700 1 -- start start 2024-04-25T14:53:27.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:27.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f143010c830 con 0x7f1430108400 2024-04-25T14:53:27.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:27.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36108/0 (socket says 172.21.15.19:36108) 2024-04-25T14:53:27.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142de00700 1 -- 172.21.15.19:0/1908163158 learned_addr learned my addr 172.21.15.19:0/1908163158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:27.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.838+0000 7f142de00700 1 -- 172.21.15.19:0/1908163158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143019eaa0 con 0x7f1430108400 2024-04-25T14:53:27.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f142de00700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1420000c00 tx=0x7f1420011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:27.053 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420003b20 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f142001a8c0 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143019ed30 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420024940 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.839+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a1a20 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.840+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1420022020 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.841+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143010c9b0 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.841+0000 7f1426a00700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 0x7f141403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.841+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f142004d8a0 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.841+0000 7f142d400700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 0x7f141403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.842+0000 7f142d400700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 0x7f141403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f141c006fd0 tx=0x7f141c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:25.846+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f142000be50 con 0x7f1430108400 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.076+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1430104d10 con 0x7f1414039600 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.286+0000 7f1426a00700 1 -- 172.21.15.19:0/1908163158 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1430104d10 con 0x7f1414039600 2024-04-25T14:53:27.054 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 msgr2=0x7f141403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 0x7f141403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f141c006fd0 tx=0x7f141c006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 msgr2=0x7f143019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1420000c00 tx=0x7f1420011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 shutdown_connections 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1414039600 0x7f141403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 --2- 172.21.15.19:0/1908163158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1430108400 0x7f143019e560 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 >> 172.21.15.19:0/1908163158 conn(0x7f1430103a00 msgr2=0x7f1430104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 shutdown_connections 2024-04-25T14:53:27.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:26.290+0000 7f142f200700 1 -- 172.21.15.19:0/1908163158 wait complete. 2024-04-25T14:53:27.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:27 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:27.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:27 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:28.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:28 smithi019 ceph-mon[28090]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:28.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:53:28.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.666+0000 7f8b93200700 1 Processor -- start 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.666+0000 7f8b93200700 1 -- start start 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.667+0000 7f8b93200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.667+0000 7f8b93200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c10b040 con 0x7f8b8c10a650 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.667+0000 7f8b91e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.667+0000 7f8b91e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36122/0 (socket says 172.21.15.19:36122) 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.667+0000 7f8b91e00700 1 -- 172.21.15.19:0/184080876 learned_addr learned my addr 172.21.15.19:0/184080876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.668+0000 7f8b91e00700 1 -- 172.21.15.19:0/184080876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c10b800 con 0x7f8b8c10a650 2024-04-25T14:53:28.478 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.668+0000 7f8b91e00700 1 --2- 172.21.15.19:0/184080876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b800098a0 tx=0x7f8b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff3247587f41042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:28.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.668+0000 7f8b90a00700 1 -- 172.21.15.19:0/184080876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80004160 con 0x7f8b8c10a650 2024-04-25T14:53:28.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b90a00700 1 -- 172.21.15.19:0/184080876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b80004300 con 0x7f8b8c10a650 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b90a00700 1 -- 172.21.15.19:0/184080876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80011af0 con 0x7f8b8c10a650 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b93200700 1 -- 172.21.15.19:0/184080876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 msgr2=0x7f8b8c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b93200700 1 --2- 172.21.15.19:0/184080876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b800098a0 tx=0x7f8b80009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b93200700 1 -- 172.21.15.19:0/184080876 shutdown_connections 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b93200700 1 --2- 172.21.15.19:0/184080876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c10aa70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.669+0000 7f8b93200700 1 -- 172.21.15.19:0/184080876 >> 172.21.15.19:0/184080876 conn(0x7f8b8c07dbf0 msgr2=0x7f8b8c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.670+0000 7f8b93200700 1 -- 172.21.15.19:0/184080876 shutdown_connections 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.670+0000 7f8b93200700 1 -- 172.21.15.19:0/184080876 wait complete. 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.671+0000 7f8b93200700 1 Processor -- start 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.671+0000 7f8b93200700 1 -- start start 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b93200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b93200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c10ea30 con 0x7f8b8c10a650 2024-04-25T14:53:28.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b91e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b91e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36124/0 (socket says 172.21.15.19:36124) 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b91e00700 1 -- 172.21.15.19:0/3953427204 learned_addr learned my addr 172.21.15.19:0/3953427204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.672+0000 7f8b91e00700 1 -- 172.21.15.19:0/3953427204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c19ea10 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b91e00700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b80011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80003b20 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b8001a8c0 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80024940 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c19eca0 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.673+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c1a1990 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.674+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8b80022020 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.674+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c10ebb0 con 0x7f8b8c10a650 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.675+0000 7f8b86a00700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 0x7f8b7403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:28.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.675+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8b8004d8a0 con 0x7f8b8c10a650 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.675+0000 7f8b91400700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 0x7f8b7403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.676+0000 7f8b91400700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 0x7f8b7403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006fd0 tx=0x7f8b7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.681+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b8000be50 con 0x7f8b8c10a650 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.913+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8b8c108300 con 0x7f8b74039600 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.914+0000 7f8b86a00700 1 -- 172.21.15.19:0/3953427204 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8b8c108300 con 0x7f8b74039600 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 msgr2=0x7f8b7403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 0x7f8b7403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006fd0 tx=0x7f8b7c006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 msgr2=0x7f8b8c19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:28.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b80011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:28.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 shutdown_connections 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f8b74039600 0x7f8b7403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 --2- 172.21.15.19:0/3953427204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b8c10a650 0x7f8b8c19e4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 >> 172.21.15.19:0/3953427204 conn(0x7f8b8c07dbf0 msgr2=0x7f8b8c107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.917+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 shutdown_connections 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:27.918+0000 7f8b93200700 1 -- 172.21.15.19:0/3953427204 wait complete. 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:53:28.487 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T14:53:28.492 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-04-25T14:53:30.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:30 smithi019 ceph-mon[28090]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:31 smithi019 ceph-mon[28090]: Deploying cephadm binary to smithi019 2024-04-25T14:53:36.501 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.135+0000 7f250fe00700 1 Processor -- start 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.135+0000 7f250fe00700 1 -- start start 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.136+0000 7f250fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.136+0000 7f250fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f250810b1f0 con 0x7f250810a800 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.136+0000 7f250d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.137+0000 7f250d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36134/0 (socket says 172.21.15.19:36134) 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.137+0000 7f250d400700 1 -- 172.21.15.19:0/2743434579 learned_addr learned my addr 172.21.15.19:0/2743434579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:36.503 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.137+0000 7f250d400700 1 -- 172.21.15.19:0/2743434579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250810b9b0 con 0x7f250810a800 2024-04-25T14:53:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.137+0000 7f250d400700 1 --2- 172.21.15.19:0/2743434579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0098a0 tx=0x7f24fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6f31a1288bc618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:36.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f2507e00700 1 -- 172.21.15.19:0/2743434579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24fc004160 con 0x7f250810a800 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f2507e00700 1 -- 172.21.15.19:0/2743434579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24fc004300 con 0x7f250810a800 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f2507e00700 1 -- 172.21.15.19:0/2743434579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24fc011af0 con 0x7f250810a800 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f250fe00700 1 -- 172.21.15.19:0/2743434579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 msgr2=0x7f250810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f250fe00700 1 --2- 172.21.15.19:0/2743434579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0098a0 tx=0x7f24fc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f250fe00700 1 -- 172.21.15.19:0/2743434579 shutdown_connections 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f250fe00700 1 --2- 172.21.15.19:0/2743434579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250810ac20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.138+0000 7f250fe00700 1 -- 172.21.15.19:0/2743434579 >> 172.21.15.19:0/2743434579 conn(0x7f250807ddf0 msgr2=0x7f25081084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.139+0000 7f250fe00700 1 -- 172.21.15.19:0/2743434579 shutdown_connections 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.139+0000 7f250fe00700 1 -- 172.21.15.19:0/2743434579 wait complete. 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.140+0000 7f250fe00700 1 Processor -- start 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.140+0000 7f250fe00700 1 -- start start 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.140+0000 7f250fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:36.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.140+0000 7f250fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f250810ebe0 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.140+0000 7f250d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f250d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36142/0 (socket says 172.21.15.19:36142) 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f250d400700 1 -- 172.21.15.19:0/2611568166 learned_addr learned my addr 172.21.15.19:0/2611568166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f250d400700 1 -- 172.21.15.19:0/2611568166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250819ebc0 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f250d400700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f24fc000c00 tx=0x7f24fc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24fc003b20 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24fc01a8c0 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.141+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24fc024940 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.142+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250819ee50 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.142+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081a1b40 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.143+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f24fc022020 con 0x7f250810a800 2024-04-25T14:53:36.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.143+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250810ed60 con 0x7f250810a800 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.143+0000 7f2506000700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 0x7f24f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.143+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f24fc04d8a0 con 0x7f250810a800 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.144+0000 7f250ca00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 0x7f24f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.144+0000 7f250ca00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 0x7f24f003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.149+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24fc00be50 con 0x7f250810a800 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:29.385+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f25081082c0 con 0x7f24f0039600 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.761+0000 7f2506000700 1 -- 172.21.15.19:0/2611568166 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f25081082c0 con 0x7f24f0039600 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.765+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 msgr2=0x7f24f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.765+0000 7f250fe00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 0x7f24f003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f4006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.765+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 msgr2=0x7f250819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.765+0000 7f250fe00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f24fc000c00 tx=0x7f24fc011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 shutdown_connections 2024-04-25T14:53:36.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f24f0039600 0x7f24f003bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 --2- 172.21.15.19:0/2611568166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250810a800 0x7f250819e680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 >> 172.21.15.19:0/2611568166 conn(0x7f250807ddf0 msgr2=0x7f2508107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 shutdown_connections 2024-04-25T14:53:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:35.766+0000 7f250fe00700 1 -- 172.21.15.19:0/2611568166 wait complete. 2024-04-25T14:53:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-04-25T14:53:37.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:36 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:37.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:36 smithi019 ceph-mon[28090]: Added host smithi019 2024-04-25T14:53:37.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:36 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:38 smithi019 ceph-mon[28090]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:38 smithi019 ceph-mon[28090]: Saving service mon spec with placement count:5 2024-04-25T14:53:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:38 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.105+0000 7fe4d8400700 1 Processor -- start 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.105+0000 7fe4d8400700 1 -- start start 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010b1f0 con 0x7fe4d010a800 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:39.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54514/0 (socket says 172.21.15.19:54514) 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d5a00700 1 -- 172.21.15.19:0/116374887 learned_addr learned my addr 172.21.15.19:0/116374887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.106+0000 7fe4d5a00700 1 -- 172.21.15.19:0/116374887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d010b9b0 con 0x7fe4d010a800 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.107+0000 7fe4d5a00700 1 --2- 172.21.15.19:0/116374887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40098a0 tx=0x7fe4c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bf94e7d2972ece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.107+0000 7fe4cfe00700 1 -- 172.21.15.19:0/116374887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4c4004160 con 0x7fe4d010a800 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.107+0000 7fe4cfe00700 1 -- 172.21.15.19:0/116374887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe4c4004300 con 0x7fe4d010a800 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.107+0000 7fe4cfe00700 1 -- 172.21.15.19:0/116374887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4c4011af0 con 0x7fe4d010a800 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 -- 172.21.15.19:0/116374887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 msgr2=0x7fe4d010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 --2- 172.21.15.19:0/116374887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40098a0 tx=0x7fe4c4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 -- 172.21.15.19:0/116374887 shutdown_connections 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 --2- 172.21.15.19:0/116374887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d010ac20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 -- 172.21.15.19:0/116374887 >> 172.21.15.19:0/116374887 conn(0x7fe4d007ddf0 msgr2=0x7fe4d01084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 -- 172.21.15.19:0/116374887 shutdown_connections 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.108+0000 7fe4d8400700 1 -- 172.21.15.19:0/116374887 wait complete. 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.109+0000 7fe4d8400700 1 Processor -- start 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.109+0000 7fe4d8400700 1 -- start start 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010ebe0 con 0x7fe4d010a800 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54530/0 (socket says 172.21.15.19:54530) 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d5a00700 1 -- 172.21.15.19:0/2644497834 learned_addr learned my addr 172.21.15.19:0/2644497834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.110+0000 7fe4d5a00700 1 -- 172.21.15.19:0/2644497834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d019ebc0 con 0x7fe4d010a800 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4d5a00700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4000c00 tx=0x7fe4c4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4c4003b20 con 0x7fe4d010a800 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe4c401a8c0 con 0x7fe4d010a800 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d019ee50 con 0x7fe4d010a800 2024-04-25T14:53:39.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4c4024940 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.111+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a1b40 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.113+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d010ed60 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.113+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe4c4022020 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.113+0000 7fe4ce000700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 0x7fe4b803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.113+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe4c404d8a0 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.114+0000 7fe4d5000700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 0x7fe4b803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.114+0000 7fe4d5000700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 0x7fe4b803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006fd0 tx=0x7fe4bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.119+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4c402c3a0 con 0x7fe4d010a800 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.410+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe4d01082c0 con 0x7fe4b8039600 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.417+0000 7fe4ce000700 1 -- 172.21.15.19:0/2644497834 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe4d01082c0 con 0x7fe4b8039600 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 msgr2=0x7fe4b803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 0x7fe4b803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006fd0 tx=0x7fe4bc006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:39.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 msgr2=0x7fe4d019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4000c00 tx=0x7fe4c4011b70 comp rx=0 tx=0).stop 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 shutdown_connections 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fe4b8039600 0x7fe4b803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 --2- 172.21.15.19:0/2644497834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d010a800 0x7fe4d019e680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.420+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 >> 172.21.15.19:0/2644497834 conn(0x7fe4d007ddf0 msgr2=0x7fe4d0107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.421+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 shutdown_connections 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:37.421+0000 7fe4d8400700 1 -- 172.21.15.19:0/2644497834 wait complete. 2024-04-25T14:53:39.337 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-04-25T14:53:40.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:39 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:40.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:39 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.895+0000 7f02cfe00700 1 Processor -- start 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.895+0000 7f02cfe00700 1 -- start start 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cfe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010afc0 con 0x7f02d010a5d0 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54546/0 (socket says 172.21.15.19:54546) 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cea00700 1 -- 172.21.15.19:0/2034702627 learned_addr learned my addr 172.21.15.19:0/2034702627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.896+0000 7f02cea00700 1 -- 172.21.15.19:0/2034702627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d010b7d0 con 0x7f02d010a5d0 2024-04-25T14:53:40.864 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.897+0000 7f02cea00700 1 --2- 172.21.15.19:0/2034702627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02c00098a0 tx=0x7f02c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63d053527514e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:40.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.897+0000 7f02cd600700 1 -- 172.21.15.19:0/2034702627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c0004160 con 0x7f02d010a5d0 2024-04-25T14:53:40.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.897+0000 7f02cd600700 1 -- 172.21.15.19:0/2034702627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c0004300 con 0x7f02d010a5d0 2024-04-25T14:53:40.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.897+0000 7f02cd600700 1 -- 172.21.15.19:0/2034702627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c0011a20 con 0x7f02d010a5d0 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 -- 172.21.15.19:0/2034702627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 msgr2=0x7f02d010a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 --2- 172.21.15.19:0/2034702627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02c00098a0 tx=0x7f02c0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 -- 172.21.15.19:0/2034702627 shutdown_connections 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 --2- 172.21.15.19:0/2034702627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d010a9f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 -- 172.21.15.19:0/2034702627 >> 172.21.15.19:0/2034702627 conn(0x7f02d007d9c0 msgr2=0x7f02d01084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 -- 172.21.15.19:0/2034702627 shutdown_connections 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.898+0000 7f02cfe00700 1 -- 172.21.15.19:0/2034702627 wait complete. 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.899+0000 7f02cfe00700 1 Processor -- start 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.899+0000 7f02cfe00700 1 -- start start 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cfe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010eb80 con 0x7f02d010a5d0 2024-04-25T14:53:40.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54562/0 (socket says 172.21.15.19:54562) 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cea00700 1 -- 172.21.15.19:0/1527581756 learned_addr learned my addr 172.21.15.19:0/1527581756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.900+0000 7f02cea00700 1 -- 172.21.15.19:0/1527581756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d019e9b0 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02cea00700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009520 tx=0x7f02c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c00039e0 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c001b8f0 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d019ec40 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02c0019770 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.901+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d01a1930 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.902+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d010ed00 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.903+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f02c001b440 con 0x7f02d010a5d0 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.903+0000 7f02c7400700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 0x7f02b403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:40.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.903+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f02c004de60 con 0x7f02d010a5d0 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.903+0000 7f02ce000700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 0x7f02b403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.904+0000 7f02ce000700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 0x7f02b403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006fd0 tx=0x7f02bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:39.909+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02c001f030 con 0x7f02d010a5d0 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.164+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f02d0108220 con 0x7f02b4039600 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.171+0000 7f02c7400700 1 -- 172.21.15.19:0/1527581756 <== mgr.14120 v2:172.21.15.19:6800/2740045356 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f02d0108220 con 0x7f02b4039600 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 msgr2=0x7f02b403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 0x7f02b403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006fd0 tx=0x7f02bc006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 msgr2=0x7f02d019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009520 tx=0x7f02c0011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 shutdown_connections 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f02b4039600 0x7f02b403bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1527581756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d010a5d0 0x7f02d019e470 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:40.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 >> 172.21.15.19:0/1527581756 conn(0x7f02d007d9c0 msgr2=0x7f02d0107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:40.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.174+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 shutdown_connections 2024-04-25T14:53:40.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:40.175+0000 7f02cfe00700 1 -- 172.21.15.19:0/1527581756 wait complete. 2024-04-25T14:53:41.465 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:41 smithi019 ceph-mon[28090]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:41.466 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:41 smithi019 ceph-mon[28090]: Saving service mgr spec with placement count:2 2024-04-25T14:53:41.466 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:41 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:42.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:42 smithi019 ceph-mon[28090]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:42.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:42 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/572824058' entity='client.admin' 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.508+0000 7f4a8dc00700 1 Processor -- start 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.508+0000 7f4a8dc00700 1 -- start start 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.509+0000 7f4a8dc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.509+0000 7f4a8dc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88063ce0 con 0x7f4a881090c0 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.509+0000 7f4a86a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.509+0000 7f4a86a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54576/0 (socket says 172.21.15.19:54576) 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.509+0000 7f4a86a00700 1 -- 172.21.15.19:0/3269837022 learned_addr learned my addr 172.21.15.19:0/3269837022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.510+0000 7f4a86a00700 1 -- 172.21.15.19:0/3269837022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8810b9f0 con 0x7f4a881090c0 2024-04-25T14:53:42.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.510+0000 7f4a86a00700 1 --2- 172.21.15.19:0/3269837022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098a0 tx=0x7f4a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692901e861d9d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:42.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.510+0000 7f4a85600700 1 -- 172.21.15.19:0/3269837022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a78004160 con 0x7f4a881090c0 2024-04-25T14:53:42.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.510+0000 7f4a85600700 1 -- 172.21.15.19:0/3269837022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a78004300 con 0x7f4a881090c0 2024-04-25T14:53:42.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.510+0000 7f4a85600700 1 -- 172.21.15.19:0/3269837022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a78011a20 con 0x7f4a881090c0 2024-04-25T14:53:42.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.511+0000 7f4a8dc00700 1 -- 172.21.15.19:0/3269837022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 msgr2=0x7f4a8810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.511+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/3269837022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098a0 tx=0x7f4a78009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.512+0000 7f4a8dc00700 1 -- 172.21.15.19:0/3269837022 shutdown_connections 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.512+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/3269837022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8810b4b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.512+0000 7f4a8dc00700 1 -- 172.21.15.19:0/3269837022 >> 172.21.15.19:0/3269837022 conn(0x7f4a88102ab0 msgr2=0x7f4a88104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.512+0000 7f4a8dc00700 1 -- 172.21.15.19:0/3269837022 shutdown_connections 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.512+0000 7f4a8dc00700 1 -- 172.21.15.19:0/3269837022 wait complete. 2024-04-25T14:53:42.641 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.513+0000 7f4a8dc00700 1 Processor -- start 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a8dc00700 1 -- start start 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a8dc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a8dc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8810c990 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a86a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a86a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54580/0 (socket says 172.21.15.19:54580) 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.514+0000 7f4a86a00700 1 -- 172.21.15.19:0/572824058 learned_addr learned my addr 172.21.15.19:0/572824058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a86a00700 1 -- 172.21.15.19:0/572824058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8819ebf0 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a86a00700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a78000c00 tx=0x7f4a78011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a78003aa0 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7801b8f0 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a78023870 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.515+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8819ee80 con 0x7f4a881090c0 2024-04-25T14:53:42.642 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.516+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881a1b70 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.517+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a880640a0 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.517+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4a7801b440 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.517+0000 7f4a7f400700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 0x7f4a6c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.518+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a7804dea0 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.518+0000 7f4a86000700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 0x7f4a6c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.519+0000 7f4a86000700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 0x7f4a6c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.523+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a7801f030 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.763+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4a881a20c0 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.766+0000 7f4a7f400700 1 -- 172.21.15.19:0/572824058 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4a7800e260 con 0x7f4a881090c0 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 msgr2=0x7f4a6c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 0x7f4a6c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 msgr2=0x7f4a8819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a78000c00 tx=0x7f4a78011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:42.643 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 shutdown_connections 2024-04-25T14:53:42.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f4a6c039600 0x7f4a6c03bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:42.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 --2- 172.21.15.19:0/572824058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a881090c0 0x7f4a8819e6b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:42.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 >> 172.21.15.19:0/572824058 conn(0x7f4a88102ab0 msgr2=0x7f4a881034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:42.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 shutdown_connections 2024-04-25T14:53:42.644 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:41.770+0000 7f4a8dc00700 1 -- 172.21.15.19:0/572824058 wait complete. 2024-04-25T14:53:44.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:44 smithi019 ceph-mon[28090]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:44.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:44 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:44.614 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3993374089' entity='client.admin' 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.642+0000 7f63eb600700 1 Processor -- start 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.643+0000 7f63eb600700 1 -- start start 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.643+0000 7f63eb600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.643+0000 7f63eb600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4063d20 con 0x7f63e4109080 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.644+0000 7f63e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.644+0000 7f63e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54594/0 (socket says 172.21.15.19:54594) 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.644+0000 7f63e8c00700 1 -- 172.21.15.19:0/966032538 learned_addr learned my addr 172.21.15.19:0/966032538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.644+0000 7f63e8c00700 1 -- 172.21.15.19:0/966032538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e410b9b0 con 0x7f63e4109080 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.644+0000 7f63e8c00700 1 --2- 172.21.15.19:0/966032538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63d80098a0 tx=0x7f63d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84e1ecba059c327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.645+0000 7f63e3400700 1 -- 172.21.15.19:0/966032538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d8004160 con 0x7f63e4109080 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.645+0000 7f63e3400700 1 -- 172.21.15.19:0/966032538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d8004300 con 0x7f63e4109080 2024-04-25T14:53:44.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.645+0000 7f63e3400700 1 -- 172.21.15.19:0/966032538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d8011a20 con 0x7f63e4109080 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 -- 172.21.15.19:0/966032538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 msgr2=0x7f63e410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 --2- 172.21.15.19:0/966032538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63d80098a0 tx=0x7f63d8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 -- 172.21.15.19:0/966032538 shutdown_connections 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 --2- 172.21.15.19:0/966032538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e410b470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 -- 172.21.15.19:0/966032538 >> 172.21.15.19:0/966032538 conn(0x7f63e4102ab0 msgr2=0x7f63e4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 -- 172.21.15.19:0/966032538 shutdown_connections 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.646+0000 7f63eb600700 1 -- 172.21.15.19:0/966032538 wait complete. 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.648+0000 7f63eb600700 1 Processor -- start 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.648+0000 7f63eb600700 1 -- start start 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.648+0000 7f63eb600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.648+0000 7f63eb600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63e410c9b0 con 0x7f63e4109080 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.649+0000 7f63e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.649+0000 7f63e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54610/0 (socket says 172.21.15.19:54610) 2024-04-25T14:53:44.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.649+0000 7f63e8c00700 1 -- 172.21.15.19:0/3993374089 learned_addr learned my addr 172.21.15.19:0/3993374089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.649+0000 7f63e8c00700 1 -- 172.21.15.19:0/3993374089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41959f0 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63e8c00700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d8003aa0 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d801b8f0 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e4195c80 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d8023870 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.650+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e4192240 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.651+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f63d801b440 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.651+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e4195e50 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.652+0000 7f63e1600700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 0x7f63cc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.652+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63d804de30 con 0x7f63e4109080 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.652+0000 7f63e3e00700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 0x7f63cc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:44.630 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.653+0000 7f63e3e00700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 0x7f63cc03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d0006fd0 tx=0x7f63d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.658+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63e4195e50 con 0x7f63e4109080 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.891+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f63e41928a0 con 0x7f63e4109080 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.895+0000 7f63e1600700 1 -- 172.21.15.19:0/3993374089 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f63d801f030 con 0x7f63e4109080 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 msgr2=0x7f63cc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 0x7f63cc03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d0006fd0 tx=0x7f63d0006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 msgr2=0x7f63e41954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 shutdown_connections 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.898+0000 7f63eb600700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f63cc039600 0x7f63cc03bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.899+0000 7f63eb600700 1 --2- 172.21.15.19:0/3993374089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63e4109080 0x7f63e41954b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.899+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 >> 172.21.15.19:0/3993374089 conn(0x7f63e4102ab0 msgr2=0x7f63e4103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.899+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 shutdown_connections 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:43.899+0000 7f63eb600700 1 -- 172.21.15.19:0/3993374089 wait complete. 2024-04-25T14:53:44.631 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-04-25T14:53:46.483 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-mon[28090]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:46.484 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-mon[28090]: from='mgr.14120 172.21.15.19:0/1161692154' entity='mgr.x' 2024-04-25T14:53:46.484 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3620871585' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T14:53:46.871 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setuser ceph since I am not root 2024-04-25T14:53:46.871 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setgroup ceph since I am not root 2024-04-25T14:53:47.140 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:46.883+0000 7f1984811200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:53:47.396 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:47.138+0000 7f1984811200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:47.396 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:47.274+0000 7f1984811200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:53:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3620871585' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T14:53:47.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-mon[28090]: mgrmap e9: x(active, since 26s) 2024-04-25T14:53:47.933 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:47.682+0000 7f1984811200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:53:47.934 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:47.803+0000 7f1984811200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.257+0000 7fa0c7600700 1 Processor -- start 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.258+0000 7fa0c7600700 1 -- start start 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.258+0000 7fa0c7600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.258+0000 7fa0c7600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0108fb0 con 0x7fa0c01085c0 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.259+0000 7fa0c4c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:48.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.259+0000 7fa0c4c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58932/0 (socket says 172.21.15.19:58932) 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.259+0000 7fa0c4c00700 1 -- 172.21.15.19:0/1627992976 learned_addr learned my addr 172.21.15.19:0/1627992976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.259+0000 7fa0c4c00700 1 -- 172.21.15.19:0/1627992976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c01097c0 con 0x7fa0c01085c0 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.260+0000 7fa0c4c00700 1 --2- 172.21.15.19:0/1627992976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098a0 tx=0x7fa0b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2fbac45c61fdac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.260+0000 7fa0bf400700 1 -- 172.21.15.19:0/1627992976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4004160 con 0x7fa0c01085c0 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.260+0000 7fa0bf400700 1 -- 172.21.15.19:0/1627992976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b4004300 con 0x7fa0c01085c0 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.260+0000 7fa0bf400700 1 -- 172.21.15.19:0/1627992976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4011a20 con 0x7fa0c01085c0 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 -- 172.21.15.19:0/1627992976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 msgr2=0x7fa0c01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 --2- 172.21.15.19:0/1627992976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098a0 tx=0x7fa0b4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:48.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 -- 172.21.15.19:0/1627992976 shutdown_connections 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 --2- 172.21.15.19:0/1627992976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c01089e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 -- 172.21.15.19:0/1627992976 >> 172.21.15.19:0/1627992976 conn(0x7fa0c0103be0 msgr2=0x7fa0c0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 -- 172.21.15.19:0/1627992976 shutdown_connections 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.261+0000 7fa0c7600700 1 -- 172.21.15.19:0/1627992976 wait complete. 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.262+0000 7fa0c7600700 1 Processor -- start 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.262+0000 7fa0c7600700 1 -- start start 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c7600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c7600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010cb70 con 0x7fa0c01085c0 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c4c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c4c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58940/0 (socket says 172.21.15.19:58940) 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c4c00700 1 -- 172.21.15.19:0/3620871585 learned_addr learned my addr 172.21.15.19:0/3620871585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.263+0000 7fa0c4c00700 1 -- 172.21.15.19:0/3620871585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c007cf80 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0c4c00700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4003aa0 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b401b8f0 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4023890 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c007b170 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.264+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c007b6b0 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.265+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa0b401b440 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.266+0000 7fa0bd600700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 0x7fa0a803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.266+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0b404de40 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.266+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c00640a0 con 0x7fa0c01085c0 2024-04-25T14:53:48.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.266+0000 7fa0bfe00700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 0x7fa0a803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.267+0000 7fa0bfe00700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 0x7fa0a803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.272+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b401f030 con 0x7fa0c01085c0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:45.590+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa0c007bfb0 con 0x7fa0c01085c0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.551+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa0b401bda0 con 0x7fa0c01085c0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.554+0000 7fa0bd600700 1 -- 172.21.15.19:0/3620871585 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa0b402c770 con 0x7fa0c01085c0 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.556+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 msgr2=0x7fa0a803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:48.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.556+0000 7fa0c7600700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 0x7fa0a803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.556+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 msgr2=0x7fa0c007ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.556+0000 7fa0c7600700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 shutdown_connections 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7fa0a80395b0 0x7fa0a803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 --2- 172.21.15.19:0/3620871585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c01085c0 0x7fa0c007ca40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 >> 172.21.15.19:0/3620871585 conn(0x7fa0c0103be0 msgr2=0x7fa0c0104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 shutdown_connections 2024-04-25T14:53:48.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:46.557+0000 7fa0c7600700 1 -- 172.21.15.19:0/3620871585 wait complete. 2024-04-25T14:53:49.580 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:49 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2361820100' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:53:49.580 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:49.432+0000 7f1984811200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:53:49.871 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:49.687+0000 7f1984811200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.900+0000 7f2604200700 1 Processor -- start 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.900+0000 7f2604200700 1 -- start start 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.901+0000 7f2604200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:49.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.901+0000 7f2604200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc063ce0 con 0x7f25fc1090c0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.901+0000 7f2601800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.901+0000 7f2601800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58968/0 (socket says 172.21.15.19:58968) 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.901+0000 7f2601800700 1 -- 172.21.15.19:0/4181796223 learned_addr learned my addr 172.21.15.19:0/4181796223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.902+0000 7f2601800700 1 -- 172.21.15.19:0/4181796223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc10b9f0 con 0x7f25fc1090c0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.902+0000 7f2601800700 1 --2- 172.21.15.19:0/4181796223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098a0 tx=0x7f25f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b2e23ddd7d7fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.902+0000 7f25fbe00700 1 -- 172.21.15.19:0/4181796223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0004160 con 0x7f25fc1090c0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.902+0000 7f25fbe00700 1 -- 172.21.15.19:0/4181796223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f0004300 con 0x7f25fc1090c0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.902+0000 7f25fbe00700 1 -- 172.21.15.19:0/4181796223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0011a20 con 0x7f25fc1090c0 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 -- 172.21.15.19:0/4181796223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 msgr2=0x7f25fc10b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 --2- 172.21.15.19:0/4181796223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098a0 tx=0x7f25f0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 -- 172.21.15.19:0/4181796223 shutdown_connections 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 --2- 172.21.15.19:0/4181796223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc10b4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 -- 172.21.15.19:0/4181796223 >> 172.21.15.19:0/4181796223 conn(0x7f25fc102ab0 msgr2=0x7f25fc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 -- 172.21.15.19:0/4181796223 shutdown_connections 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.903+0000 7f2604200700 1 -- 172.21.15.19:0/4181796223 wait complete. 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.904+0000 7f2604200700 1 Processor -- start 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.904+0000 7f2604200700 1 -- start start 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2604200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2604200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10c990 con 0x7f25fc1090c0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2601800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2601800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58974/0 (socket says 172.21.15.19:58974) 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2601800700 1 -- 172.21.15.19:0/2361820100 learned_addr learned my addr 172.21.15.19:0/2361820100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.905+0000 7f2601800700 1 -- 172.21.15.19:0/2361820100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc19eb60 con 0x7f25fc1090c0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f2601800700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25f0000c00 tx=0x7f25f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0003aa0 con 0x7f25fc1090c0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f001b8f0 con 0x7f25fc1090c0 2024-04-25T14:53:49.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0023870 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc19edf0 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.906+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a1ae0 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.907+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0640a0 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.907+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f25f001b440 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.908+0000 7f25fa000700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 0x7f25e403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.908+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f25f004de90 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.908+0000 7f2600e00700 1 -- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 msgr2=0x7f25e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.908+0000 7f2600e00700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 0x7f25e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:48.914+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25f001f030 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.109+0000 7f2600e00700 1 -- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 msgr2=0x7f25e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.109+0000 7f2600e00700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 0x7f25e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.202+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f25fc1a2020 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.203+0000 7f25fa000700 1 -- 172.21.15.19:0/2361820100 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f25f000be50 con 0x7f25fc1090c0 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 msgr2=0x7f25e403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 0x7f25e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 msgr2=0x7f25fc19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25f0000c00 tx=0x7f25f0011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 shutdown_connections 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f25e4039660 0x7f25e403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 --2- 172.21.15.19:0/2361820100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25fc1090c0 0x7f25fc19e620 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.207+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 >> 172.21.15.19:0/2361820100 conn(0x7f25fc102ab0 msgr2=0x7f25fc1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.208+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 shutdown_connections 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:49.208+0000 7f2604200700 1 -- 172.21.15.19:0/2361820100 wait complete. 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T14:53:49.966 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-04-25T14:53:50.266 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:49.908+0000 7f1984811200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:53:50.517 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:50.312+0000 7f1984811200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:53:50.517 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:50.439+0000 7f1984811200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:53:50.800 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:50.702+0000 7f1984811200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:53:51.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:50.966+0000 7f1984811200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:52.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:51.776+0000 7f1984811200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:53:52.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:52 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:52.347+0000 7f1984811200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:53:52.621 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:52 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:52.478+0000 7f1984811200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:53:53.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:52 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:52.754+0000 7f1984811200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:53:53.819 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:53.429+0000 7f1984811200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:53:53.819 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:53.560+0000 7f1984811200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:53:54.092 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:53.901+0000 7f1984811200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:53:54.092 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:54 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:54.090+0000 7f1984811200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:53:54.371 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:54 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:54.222+0000 7f1984811200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:53:54.741 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:54 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:54.382+0000 7f1984811200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:55.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:54 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:54.763+0000 7f1984811200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:53:55.849 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:55.553+0000 7f1984811200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:53:55.848+0000 7f1984811200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: Active manager daemon x restarted 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: Activating manager daemon x 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: mgrmap e10: x(active, starting, since 0.00908369s) 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:53:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:53:56.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:53:56.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:55 smithi019 ceph-mon[28090]: Manager daemon x is now available 2024-04-25T14:53:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:53:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:53:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:56 smithi019 ceph-mon[28090]: mgrmap e11: x(active, since 1.01159s) 2024-04-25T14:53:57.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.589+0000 7f1e7a800700 1 Processor -- start 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.589+0000 7f1e7a800700 1 -- start start 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.589+0000 7f1e7a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.589+0000 7f1e7a800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74108fc0 con 0x7f1e741085d0 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.590+0000 7f1e73400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.585 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.590+0000 7f1e73400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58978/0 (socket says 172.21.15.19:58978) 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.590+0000 7f1e73400700 1 -- 172.21.15.19:0/1478606356 learned_addr learned my addr 172.21.15.19:0/1478606356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.590+0000 7f1e73400700 1 -- 172.21.15.19:0/1478606356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e74109780 con 0x7f1e741085d0 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.591+0000 7f1e73400700 1 --2- 172.21.15.19:0/1478606356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e640098a0 tx=0x7f1e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65f17b9e0a3d206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.591+0000 7f1e72a00700 1 -- 172.21.15.19:0/1478606356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64004160 con 0x7f1e741085d0 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.591+0000 7f1e72a00700 1 -- 172.21.15.19:0/1478606356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e64004300 con 0x7f1e741085d0 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.591+0000 7f1e72a00700 1 -- 172.21.15.19:0/1478606356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64011a20 con 0x7f1e741085d0 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 -- 172.21.15.19:0/1478606356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 msgr2=0x7f1e741089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 --2- 172.21.15.19:0/1478606356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e640098a0 tx=0x7f1e64009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 -- 172.21.15.19:0/1478606356 shutdown_connections 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 --2- 172.21.15.19:0/1478606356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e741089f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.586 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 -- 172.21.15.19:0/1478606356 >> 172.21.15.19:0/1478606356 conn(0x7f1e74103bd0 msgr2=0x7f1e74105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 -- 172.21.15.19:0/1478606356 shutdown_connections 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.592+0000 7f1e7a800700 1 -- 172.21.15.19:0/1478606356 wait complete. 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.593+0000 7f1e7a800700 1 Processor -- start 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e7a800700 1 -- start start 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e7a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e7a800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7407c0c0 con 0x7f1e741085d0 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e73400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e73400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58990/0 (socket says 172.21.15.19:58990) 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.594+0000 7f1e73400700 1 -- 172.21.15.19:0/3674965719 learned_addr learned my addr 172.21.15.19:0/3674965719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.595+0000 7f1e73400700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7419ebc0 con 0x7f1e741085d0 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.595+0000 7f1e73400700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009520 tx=0x7f1e64011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.596+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e640039e0 con 0x7f1e741085d0 2024-04-25T14:53:57.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.596+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6401b8f0 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.596+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7419ee50 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.596+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64019770 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.596+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e741a1b40 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1e6401b440 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e70c00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e5803c220 con 0x7f1e58039650 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1e6404dfa0 con 0x7f1e741085d0 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e6aa00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.598+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.799+0000 7f1e6aa00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:50.799+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:51.200+0000 7f1e6aa00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:57.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:51.200+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:52.001+0000 7f1e6aa00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:52.001+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:53.603+0000 7f1e6aa00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2740045356 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:53.603+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:55.858+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f1e6401a3d0 con 0x7f1e741085d0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:55.858+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 msgr2=0x7f1e5803bb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:55.858+0000 7f1e70c00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.862+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f1e6404fc40 con 0x7f1e741085d0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.862+0000 7f1e70c00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 0x7f1e5803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.862+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e6404c460 con 0x7f1e5803cfe0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.865+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 0x7f1e5803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.867+0000 7f1e6aa00700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 0x7f1e5803f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c003a10 tx=0x7f1e5c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.869+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f1e6404c460 con 0x7f1e5803cfe0 2024-04-25T14:53:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.882+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1e7419f330 con 0x7f1e5803cfe0 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.882+0000 7f1e70c00700 1 -- 172.21.15.19:0/3674965719 <== mgr.14152 v2:172.21.15.19:6800/983233775 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1e7419f330 con 0x7f1e5803cfe0 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 msgr2=0x7f1e5803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 0x7f1e5803f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c003a10 tx=0x7f1e5c009270 comp rx=0 tx=0).stop 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 msgr2=0x7f1e7419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009520 tx=0x7f1e64011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 shutdown_connections 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1e5803cfe0 0x7f1e5803f3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e741085d0 0x7f1e7419e680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 --2- 172.21.15.19:0/3674965719 >> [v2:172.21.15.19:6800/2740045356,v1:172.21.15.19:6801/2740045356] conn(0x7f1e58039650 0x7f1e5803bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 >> 172.21.15.19:0/3674965719 conn(0x7f1e74103bd0 msgr2=0x7f1e74105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 shutdown_connections 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:56.883+0000 7f1e7a800700 1 -- 172.21.15.19:0/3674965719 wait complete. 2024-04-25T14:53:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-04-25T14:53:57.591 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-04-25T14:53:58.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:57 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:57] ENGINE Bus STARTING 2024-04-25T14:53:58.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:57 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:57] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T14:53:58.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:57 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:57] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T14:53:58.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:57 smithi019 ceph-mon[28090]: [25/Apr/2024:14:53:57] ENGINE Bus STARTED 2024-04-25T14:53:59.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:58 smithi019 ceph-mon[28090]: mgrmap e12: x(active, since 2s) 2024-04-25T14:53:59.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:58 smithi019 ceph-mon[28090]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:59.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:53:59.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:53:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:53:59.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.329+0000 7fc39ea00700 1 Processor -- start 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.329+0000 7fc39ea00700 1 -- start start 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc39ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc398108fd0 con 0x7fc3981085e0 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc397e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc397e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60882/0 (socket says 172.21.15.19:60882) 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc397e00700 1 -- 172.21.15.19:0/2239754387 learned_addr learned my addr 172.21.15.19:0/2239754387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.330+0000 7fc397e00700 1 -- 172.21.15.19:0/2239754387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc398109790 con 0x7fc3981085e0 2024-04-25T14:53:59.599 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.331+0000 7fc397e00700 1 --2- 172.21.15.19:0/2239754387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1260138b79c65d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:59.600 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.331+0000 7fc396a00700 1 -- 172.21.15.19:0/2239754387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388004160 con 0x7fc3981085e0 2024-04-25T14:53:59.600 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.331+0000 7fc396a00700 1 -- 172.21.15.19:0/2239754387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc388004300 con 0x7fc3981085e0 2024-04-25T14:53:59.604 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.331+0000 7fc396a00700 1 -- 172.21.15.19:0/2239754387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388011a20 con 0x7fc3981085e0 2024-04-25T14:53:59.604 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 -- 172.21.15.19:0/2239754387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 msgr2=0x7fc398108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:59.604 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 --2- 172.21.15.19:0/2239754387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).stop 2024-04-25T14:53:59.604 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 -- 172.21.15.19:0/2239754387 shutdown_connections 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 --2- 172.21.15.19:0/2239754387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc398108a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 -- 172.21.15.19:0/2239754387 >> 172.21.15.19:0/2239754387 conn(0x7fc398103c60 msgr2=0x7fc398106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 -- 172.21.15.19:0/2239754387 shutdown_connections 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.332+0000 7fc39ea00700 1 -- 172.21.15.19:0/2239754387 wait complete. 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.333+0000 7fc39ea00700 1 Processor -- start 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.333+0000 7fc39ea00700 1 -- start start 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc39ea00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810cb40 con 0x7fc3981085e0 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc397e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc397e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60886/0 (socket says 172.21.15.19:60886) 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc397e00700 1 -- 172.21.15.19:0/4209192891 learned_addr learned my addr 172.21.15.19:0/4209192891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.334+0000 7fc397e00700 1 -- 172.21.15.19:0/4209192891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39819c9d0 con 0x7fc3981085e0 2024-04-25T14:53:59.605 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc397e00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388003aa0 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38801b8f0 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388023890 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39819cc60 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.335+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39819f950 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.336+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc38801b440 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.336+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39810ccc0 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.337+0000 7fc394c00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 0x7fc37c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.337+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc38804e010 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.337+0000 7fc397400700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 0x7fc37c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.338+0000 7fc397400700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 0x7fc37c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.343+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc38801f030 con 0x7fc3981085e0 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.579+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fc398104f20 con 0x7fc37c039660 2024-04-25T14:53:59.606 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.769+0000 7fc394c00700 1 -- 172.21.15.19:0/4209192891 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fc398104f20 con 0x7fc37c039660 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 msgr2=0x7fc37c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 0x7fc37c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).stop 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 msgr2=0x7fc39819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388011aa0 comp rx=0 tx=0).stop 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 shutdown_connections 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc37c039660 0x7fc37c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 --2- 172.21.15.19:0/4209192891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3981085e0 0x7fc39819c490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 >> 172.21.15.19:0/4209192891 conn(0x7fc398103c60 msgr2=0x7fc398104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.772+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 shutdown_connections 2024-04-25T14:53:59.607 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:53:58.773+0000 7fc39ea00700 1 -- 172.21.15.19:0/4209192891 wait complete. 2024-04-25T14:53:59.608 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-04-25T14:54:00.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:01.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:01 smithi019 ceph-mon[28090]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:01.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$bbLZeogi5co/66B/wcYuK./CyGCs9MwwqRRDu8YF8oW.DslRXjj/a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714056840, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.279+0000 7f4a83e00700 1 Processor -- start 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.279+0000 7f4a83e00700 1 -- start start 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.279+0000 7f4a83e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.279+0000 7f4a83e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8407b780 con 0x7f4a8407cd50 2024-04-25T14:54:01.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.280+0000 7f4a82a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.280+0000 7f4a82a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60890/0 (socket says 172.21.15.19:60890) 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.280+0000 7f4a82a00700 1 -- 172.21.15.19:0/3209310782 learned_addr learned my addr 172.21.15.19:0/3209310782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.281+0000 7f4a82a00700 1 -- 172.21.15.19:0/3209310782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8407b900 con 0x7f4a8407cd50 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.281+0000 7f4a82a00700 1 --2- 172.21.15.19:0/3209310782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a740098a0 tx=0x7f4a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c256eca6a855d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.281+0000 7f4a81600700 1 -- 172.21.15.19:0/3209310782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a74004160 con 0x7f4a8407cd50 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.281+0000 7f4a81600700 1 -- 172.21.15.19:0/3209310782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74004300 con 0x7f4a8407cd50 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.281+0000 7f4a81600700 1 -- 172.21.15.19:0/3209310782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a74011a20 con 0x7f4a8407cd50 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 -- 172.21.15.19:0/3209310782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 msgr2=0x7f4a8407b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 --2- 172.21.15.19:0/3209310782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a740098a0 tx=0x7f4a74009bb0 comp rx=0 tx=0).stop 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 -- 172.21.15.19:0/3209310782 shutdown_connections 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 --2- 172.21.15.19:0/3209310782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8407b1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 -- 172.21.15.19:0/3209310782 >> 172.21.15.19:0/3209310782 conn(0x7f4a84103a00 msgr2=0x7f4a84105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:01.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 -- 172.21.15.19:0/3209310782 shutdown_connections 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.282+0000 7f4a83e00700 1 -- 172.21.15.19:0/3209310782 wait complete. 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.283+0000 7f4a83e00700 1 Processor -- start 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.283+0000 7f4a83e00700 1 -- start start 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a83e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a83e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8410ea20 con 0x7f4a8407cd50 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a82a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a82a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60902/0 (socket says 172.21.15.19:60902) 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a82a00700 1 -- 172.21.15.19:0/2411676133 learned_addr learned my addr 172.21.15.19:0/2411676133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.284+0000 7f4a82a00700 1 -- 172.21.15.19:0/2411676133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8419e9c0 con 0x7f4a8407cd50 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a82a00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a74011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a74003aa0 con 0x7f4a8407cd50 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7401b8f0 con 0x7f4a8407cd50 2024-04-25T14:54:01.556 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8419ec50 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a74023870 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.285+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841a1940 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.287+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84069b80 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.287+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4a74023b70 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.287+0000 7f4a88a00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 0x7f4a6803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.288+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4a7404e070 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.288+0000 7f4a82000700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 0x7f4a6803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.289+0000 7f4a82000700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 0x7f4a6803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006fd0 tx=0x7f4a70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.298+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a7401f030 con 0x7f4a8407cd50 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.548+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4a8419f130 con 0x7f4a68039600 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.833+0000 7f4a88a00700 1 -- 172.21.15.19:0/2411676133 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4a8419f130 con 0x7f4a68039600 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 msgr2=0x7f4a6803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 0x7f4a6803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006fd0 tx=0x7f4a70006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 msgr2=0x7f4a8419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a74011aa0 comp rx=0 tx=0).stop 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 shutdown_connections 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4a68039600 0x7f4a6803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 --2- 172.21.15.19:0/2411676133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8407cd50 0x7f4a8419e480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 >> 172.21.15.19:0/2411676133 conn(0x7f4a84103a00 msgr2=0x7f4a841044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 shutdown_connections 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:00.836+0000 7f4a83e00700 1 -- 172.21.15.19:0/2411676133 wait complete. 2024-04-25T14:54:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-04-25T14:54:02.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:02.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:02 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3666702027' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T14:54:03.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T14:54:03.248 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.163+0000 7fbcec600700 1 Processor -- start 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.163+0000 7fbcec600700 1 -- start start 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.163+0000 7fbcec600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.164+0000 7fbcec600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbce407b7a0 con 0x7fbce407cd70 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.164+0000 7fbce9c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.164+0000 7fbce9c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60906/0 (socket says 172.21.15.19:60906) 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.164+0000 7fbce9c00700 1 -- 172.21.15.19:0/2699800028 learned_addr learned my addr 172.21.15.19:0/2699800028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.165+0000 7fbce9c00700 1 -- 172.21.15.19:0/2699800028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce407b920 con 0x7fbce407cd70 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.165+0000 7fbce9c00700 1 --2- 172.21.15.19:0/2699800028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9507ccf792264a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.165+0000 7fbce3e00700 1 -- 172.21.15.19:0/2699800028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8004160 con 0x7fbce407cd70 2024-04-25T14:54:03.250 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.166+0000 7fbce3e00700 1 -- 172.21.15.19:0/2699800028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8004300 con 0x7fbce407cd70 2024-04-25T14:54:03.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.166+0000 7fbce3e00700 1 -- 172.21.15.19:0/2699800028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8011a20 con 0x7fbce407cd70 2024-04-25T14:54:03.251 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.166+0000 7fbcec600700 1 -- 172.21.15.19:0/2699800028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 msgr2=0x7fbce407b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.166+0000 7fbcec600700 1 --2- 172.21.15.19:0/2699800028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.167+0000 7fbcec600700 1 -- 172.21.15.19:0/2699800028 shutdown_connections 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.167+0000 7fbcec600700 1 --2- 172.21.15.19:0/2699800028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce407b1d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.167+0000 7fbcec600700 1 -- 172.21.15.19:0/2699800028 >> 172.21.15.19:0/2699800028 conn(0x7fbce4103be0 msgr2=0x7fbce4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.167+0000 7fbcec600700 1 -- 172.21.15.19:0/2699800028 shutdown_connections 2024-04-25T14:54:03.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.167+0000 7fbcec600700 1 -- 172.21.15.19:0/2699800028 wait complete. 2024-04-25T14:54:03.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.168+0000 7fbcec600700 1 Processor -- start 2024-04-25T14:54:03.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.168+0000 7fbcec600700 1 -- start start 2024-04-25T14:54:03.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.168+0000 7fbcec600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:03.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.168+0000 7fbcec600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410ebc0 con 0x7fbce407cd70 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.169+0000 7fbce9c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.169+0000 7fbce9c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60922/0 (socket says 172.21.15.19:60922) 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.169+0000 7fbce9c00700 1 -- 172.21.15.19:0/3666702027 learned_addr learned my addr 172.21.15.19:0/3666702027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.169+0000 7fbce9c00700 1 -- 172.21.15.19:0/3666702027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce419ebe0 con 0x7fbce407cd70 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.169+0000 7fbce9c00700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8000c00 tx=0x7fbcd8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.170+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8003aa0 con 0x7fbce407cd70 2024-04-25T14:54:03.255 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.170+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd801b8f0 con 0x7fbce407cd70 2024-04-25T14:54:03.256 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.170+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce419ee70 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.170+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8023890 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.170+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce41a1b60 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.171+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce40640a0 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.172+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fbcd801b440 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.173+0000 7fbce2000700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 0x7fbccc03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.173+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbcd804e170 con 0x7fbce407cd70 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.173+0000 7fbce9200700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 0x7fbccc03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.174+0000 7fbce9200700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 0x7fbccc03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:03.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.177+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcd801f030 con 0x7fbce407cd70 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.401+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fbce41a1d30 con 0x7fbce407cd70 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.402+0000 7fbce2000700 1 -- 172.21.15.19:0/3666702027 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fbcd801a3d0 con 0x7fbce407cd70 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.404+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 msgr2=0x7fbccc03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.404+0000 7fbcec600700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 0x7fbccc03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0006fd0 tx=0x7fbcd0006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 msgr2=0x7fbce419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8000c00 tx=0x7fbcd8011aa0 comp rx=0 tx=0).stop 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 shutdown_connections 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbccc039660 0x7fbccc03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 --2- 172.21.15.19:0/3666702027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbce407cd70 0x7fbce419e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:03.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 >> 172.21.15.19:0/3666702027 conn(0x7fbce4103be0 msgr2=0x7fbce4104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:03.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 shutdown_connections 2024-04-25T14:54:03.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:02.405+0000 7fbcec600700 1 -- 172.21.15.19:0/3666702027 wait complete. 2024-04-25T14:54:03.300 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:54:03.300 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T14:54:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:54:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T14:54:03.329 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T14:54:03.330 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout: Password: yenyeo0f4e 2024-04-25T14:54:03.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:03.338 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config directory 2024-04-25T14:54:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-04-25T14:54:03.863 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:03 smithi019 ceph-mon[28090]: mgrmap e13: x(active, since 6s) 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.226+0000 7f6196800700 1 Processor -- start 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.226+0000 7f6196800700 1 -- start start 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.227+0000 7f6196800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.227+0000 7f6196800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f619007c7e0 con 0x7f619007ddf0 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.227+0000 7f618f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.227+0000 7f618f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37600/0 (socket says 172.21.15.19:37600) 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.228+0000 7f618f400700 1 -- 172.21.15.19:0/2484933918 learned_addr learned my addr 172.21.15.19:0/2484933918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.228+0000 7f618f400700 1 -- 172.21.15.19:0/2484933918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619007c960 con 0x7f619007ddf0 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.228+0000 7f618f400700 1 --2- 172.21.15.19:0/2484933918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61800098a0 tx=0x7f6180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4afdb8a98300ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.229+0000 7f618e000700 1 -- 172.21.15.19:0/2484933918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180004160 con 0x7f619007ddf0 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.229+0000 7f618e000700 1 -- 172.21.15.19:0/2484933918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6180004300 con 0x7f619007ddf0 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.229+0000 7f618e000700 1 -- 172.21.15.19:0/2484933918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180011a20 con 0x7f619007ddf0 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.229+0000 7f6196800700 1 -- 172.21.15.19:0/2484933918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 msgr2=0x7f619007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.229+0000 7f6196800700 1 --2- 172.21.15.19:0/2484933918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61800098a0 tx=0x7f6180009bb0 comp rx=0 tx=0).stop 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.230+0000 7f6196800700 1 -- 172.21.15.19:0/2484933918 shutdown_connections 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.230+0000 7f6196800700 1 --2- 172.21.15.19:0/2484933918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619007c2a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.230+0000 7f6196800700 1 -- 172.21.15.19:0/2484933918 >> 172.21.15.19:0/2484933918 conn(0x7f6190103b40 msgr2=0x7f6190105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.230+0000 7f6196800700 1 -- 172.21.15.19:0/2484933918 shutdown_connections 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.230+0000 7f6196800700 1 -- 172.21.15.19:0/2484933918 wait complete. 2024-04-25T14:54:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.231+0000 7f6196800700 1 Processor -- start 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.231+0000 7f6196800700 1 -- start start 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.231+0000 7f6196800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.231+0000 7f6196800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f619010eb20 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.232+0000 7f618f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.232+0000 7f618f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37602/0 (socket says 172.21.15.19:37602) 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.232+0000 7f618f400700 1 -- 172.21.15.19:0/976072174 learned_addr learned my addr 172.21.15.19:0/976072174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.232+0000 7f618f400700 1 -- 172.21.15.19:0/976072174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619019eb80 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.232+0000 7f618f400700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61800116e0 tx=0x7f6180011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.233+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180003a60 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.233+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618001b8f0 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.233+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180019860 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.233+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619019ee10 con 0x7f619007ddf0 2024-04-25T14:54:05.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.233+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901a1ab0 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.234+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61900640a0 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.234+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f618001b440 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.235+0000 7f6187e00700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 0x7f617403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.235+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f618004df60 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.235+0000 7f618ea00700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 0x7f617403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.236+0000 7f618ea00700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 0x7f617403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f617c006fd0 tx=0x7f617c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.240+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f618001f030 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.471+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f61901a1c80 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.472+0000 7f6187e00700 1 -- 172.21.15.19:0/976072174 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f61800223b0 con 0x7f619007ddf0 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.475+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 msgr2=0x7f617403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:05.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.475+0000 7f6196800700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 0x7f617403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f617c006fd0 tx=0x7f617c006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.475+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 msgr2=0x7f619019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.475+0000 7f6196800700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61800116e0 tx=0x7f6180011aa0 comp rx=0 tx=0).stop 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 shutdown_connections 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f61740396b0 0x7f617403bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 --2- 172.21.15.19:0/976072174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619007ddf0 0x7f619019e640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 >> 172.21.15.19:0/976072174 conn(0x7f6190103b40 msgr2=0x7f6190104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 shutdown_connections 2024-04-25T14:54:05.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:04.476+0000 7f6196800700 1 -- 172.21.15.19:0/976072174 wait complete. 2024-04-25T14:54:07.252 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4bbe00700 1 Processor -- start 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4bbe00700 1 -- start start 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4bbe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4bbe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc108e00 con 0x7fe4bc108410 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37618/0 (socket says 172.21.15.19:37618) 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4baa00700 1 -- 172.21.15.19:0/735820362 learned_addr learned my addr 172.21.15.19:0/735820362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.142+0000 7fe4baa00700 1 -- 172.21.15.19:0/735820362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc1095c0 con 0x7fe4bc108410 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.143+0000 7fe4baa00700 1 --2- 172.21.15.19:0/735820362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b642c7f6dbfbaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.143+0000 7fe4ba000700 1 -- 172.21.15.19:0/735820362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac004160 con 0x7fe4bc108410 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.143+0000 7fe4ba000700 1 -- 172.21.15.19:0/735820362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac004300 con 0x7fe4bc108410 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.143+0000 7fe4ba000700 1 -- 172.21.15.19:0/735820362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac011a20 con 0x7fe4bc108410 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 -- 172.21.15.19:0/735820362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 msgr2=0x7fe4bc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:07.253 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/735820362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).stop 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 -- 172.21.15.19:0/735820362 shutdown_connections 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/735820362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc108830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 -- 172.21.15.19:0/735820362 >> 172.21.15.19:0/735820362 conn(0x7fe4bc103a50 msgr2=0x7fe4bc105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 -- 172.21.15.19:0/735820362 shutdown_connections 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.144+0000 7fe4bbe00700 1 -- 172.21.15.19:0/735820362 wait complete. 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.145+0000 7fe4bbe00700 1 Processor -- start 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.145+0000 7fe4bbe00700 1 -- start start 2024-04-25T14:54:07.254 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4bbe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:07.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4bbe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc10c970 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37634/0 (socket says 172.21.15.19:37634) 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4baa00700 1 -- 172.21.15.19:0/275310581 learned_addr learned my addr 172.21.15.19:0/275310581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.146+0000 7fe4baa00700 1 -- 172.21.15.19:0/275310581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc19ea00 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4baa00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0119f0 tx=0x7fe4ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac0039e0 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac01b8f0 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac0237a0 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc19ec90 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.147+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc1a1980 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.148+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe4ac01b440 con 0x7fe4bc108410 2024-04-25T14:54:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.148+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4bc10caf0 con 0x7fe4bc108410 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.149+0000 7fe4b3400700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 0x7fe4a003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.149+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe4ac04e120 con 0x7fe4bc108410 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.149+0000 7fe4b3e00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 0x7fe4a003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.150+0000 7fe4b3e00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 0x7fe4a003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.154+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4ac01f030 con 0x7fe4bc108410 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.523+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe4bc1a1b50 con 0x7fe4bc108410 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.528+0000 7fe4b3400700 1 -- 172.21.15.19:0/275310581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe4ac0202b0 con 0x7fe4bc108410 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 msgr2=0x7fe4a003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 0x7fe4a003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 msgr2=0x7fe4bc19e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0119f0 tx=0x7fe4ac011ad0 comp rx=0 tx=0).stop 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 shutdown_connections 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4a0039650 0x7fe4a003bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 --2- 172.21.15.19:0/275310581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4bc108410 0x7fe4bc19e4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 >> 172.21.15.19:0/275310581 conn(0x7fe4bc103a50 msgr2=0x7fe4bc104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 shutdown_connections 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T14:54:06.531+0000 7fe4bbe00700 1 -- 172.21.15.19:0/275310581 wait complete. 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T14:54:07.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-04-25T14:54:07.261 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.262 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-04-25T14:54:07.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.262 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T14:54:07.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:07.262 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-04-25T14:54:07.371 INFO:tasks.cephadm:Fetching config... 2024-04-25T14:54:07.372 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:54:07.372 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T14:54:07.424 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T14:54:07.424 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:54:07.424 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T14:54:07.467 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T14:54:07.467 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:54:07.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T14:54:07.579 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T14:54:07.579 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:54:07.579 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T14:54:07.671 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T14:54:07.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:54:07.779 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:07.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:07 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/275310581' entity='client.admin' 2024-04-25T14:54:07.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:54:07.857 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:07.869 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:54:07.910 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9UhhosVcZFnDWLteOFkCr9EhvHuopXa17n0EScXn57IjawSXOY+jg9AYTKGEdIFfKA/8kz4LKijHBRZZZViWVVVFzO4UQ4YTrGnhp2dLiSutpC9x1kF/3KVyTWN83Dyo1jz+iYLVaicbORTw7CJTMoBgpJBIe8cvJCQ0lzW/hUzHdujAI4RJthVVvB5oSixqW7x7PnIvHDadCDf0cY1wgNk39/8xPAzz+ykk/ow5uDhqHnQtZNjHEXgzUcqHcm75ZkFB+FtXcR6VrnqNH1jTiKG5sG4rNLdfaGCNtU+zIfD2xbA7wgk3KZj+LJM713cefShg7vQ0BGjKNGPgxccgjOn6deV5D5aRVxQoAfUWvANv/gr4Jn9kP9/GSypqkeoo4I6TLNHH5WL7g73HnMSoWVOOURQcK4+s0HaoV0iS9qtEvOBsJAfds/ny30GohlM/y8QkHdX8l0gLd04X0dbh93cI47E+nPDNY1WMBoKpx7o2AHoVUcG40phm93oFhCtk= ceph-42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:07.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T14:54:08.371 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:11.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 -- 172.21.15.19:0/4049306765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 msgr2=0x7f04e4074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 --2- 172.21.15.19:0/4049306765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e4074ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009910 tx=0x7f04d8009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 -- 172.21.15.19:0/4049306765 shutdown_connections 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 --2- 172.21.15.19:0/4049306765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e4074ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 -- 172.21.15.19:0/4049306765 >> 172.21.15.19:0/4049306765 conn(0x7f04e40fda60 msgr2=0x7f04e40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 -- 172.21.15.19:0/4049306765 shutdown_connections 2024-04-25T14:54:11.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.103+0000 7f04ec800700 1 -- 172.21.15.19:0/4049306765 wait complete. 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.104+0000 7f04ec800700 1 Processor -- start 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04ec800700 1 -- start start 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04ec800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04ec800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410eaa0 con 0x7f04e4076a70 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04e9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04e9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37646/0 (socket says 172.21.15.19:37646) 2024-04-25T14:54:11.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.105+0000 7f04e9e00700 1 -- 172.21.15.19:0/3269960806 learned_addr learned my addr 172.21.15.19:0/3269960806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:11.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.106+0000 7f04e9e00700 1 -- 172.21.15.19:0/3269960806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419e640 con 0x7f04e4076a70 2024-04-25T14:54:11.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.106+0000 7f04e9e00700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005950 tx=0x7f04d8004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.106+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8026070 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.106+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d800bc00 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.106+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e419e8d0 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.107+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a15c0 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.107+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8004830 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.108+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e410f180 con 0x7f04e4076a70 2024-04-25T14:54:11.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.108+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f04d8029070 con 0x7f04e4076a70 2024-04-25T14:54:11.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.108+0000 7f04dea00700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 0x7f04cc03bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:11.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.109+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f04d801e070 con 0x7f04e4076a70 2024-04-25T14:54:11.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.109+0000 7f04e9400700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 0x7f04cc03bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:11.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.110+0000 7f04e9400700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 0x7f04cc03bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006fd0 tx=0x7f04d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:11.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.114+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04d8013070 con 0x7f04e4076a70 2024-04-25T14:54:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.352+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f04e402dff0 con 0x7f04e4076a70 2024-04-25T14:54:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.356+0000 7f04dea00700 1 -- 172.21.15.19:0/3269960806 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f04d80300c0 con 0x7f04e4076a70 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 msgr2=0x7f04cc03bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 0x7f04cc03bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006fd0 tx=0x7f04d4006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 msgr2=0x7f04e419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005950 tx=0x7f04d8004fa0 comp rx=0 tx=0).stop 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 shutdown_connections 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f04cc0397f0 0x7f04cc03bcb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 --2- 172.21.15.19:0/3269960806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04e4076a70 0x7f04e419e100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 >> 172.21.15.19:0/3269960806 conn(0x7f04e40fda60 msgr2=0x7f04e40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 shutdown_connections 2024-04-25T14:54:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:11.359+0000 7f04ec800700 1 -- 172.21.15.19:0/3269960806 wait complete. 2024-04-25T14:54:12.355 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T14:54:12.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T14:54:12.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:12 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3269960806' entity='client.admin' 2024-04-25T14:54:12.779 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 -- 172.21.15.19:0/41005751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 msgr2=0x7fb4dc074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 --2- 172.21.15.19:0/41005751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc074ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009910 tx=0x7fb4d0009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 -- 172.21.15.19:0/41005751 shutdown_connections 2024-04-25T14:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 --2- 172.21.15.19:0/41005751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc074ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 -- 172.21.15.19:0/41005751 >> 172.21.15.19:0/41005751 conn(0x7fb4dc0fd8f0 msgr2=0x7fb4dc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 -- 172.21.15.19:0/41005751 shutdown_connections 2024-04-25T14:54:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.583+0000 7fb4e2800700 1 -- 172.21.15.19:0/41005751 wait complete. 2024-04-25T14:54:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.584+0000 7fb4e2800700 1 Processor -- start 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.584+0000 7fb4e2800700 1 -- start start 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e2800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc10e8d0 con 0x7fb4dc076a70 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e1400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e1400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60498/0 (socket says 172.21.15.19:60498) 2024-04-25T14:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e1400700 1 -- 172.21.15.19:0/31020636 learned_addr learned my addr 172.21.15.19:0/31020636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.585+0000 7fb4e1400700 1 -- 172.21.15.19:0/31020636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc19e4a0 con 0x7fb4dc076a70 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4e1400700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005950 tx=0x7fb4d0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d0026070 con 0x7fb4dc076a70 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d000bc00 con 0x7fb4dc076a70 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc19e730 con 0x7fb4dc076a70 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d0003a10 con 0x7fb4dc076a70 2024-04-25T14:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.586+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1a1420 con 0x7fb4dc076a70 2024-04-25T14:54:15.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.587+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4dc10efb0 con 0x7fb4dc076a70 2024-04-25T14:54:15.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.587+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb4d0029070 con 0x7fb4dc076a70 2024-04-25T14:54:15.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.588+0000 7fb4da000700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 0x7fb4c403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.588+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4d001e070 con 0x7fb4dc076a70 2024-04-25T14:54:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.588+0000 7fb4e0a00700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 0x7fb4c403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:15.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.589+0000 7fb4e0a00700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 0x7fb4c403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8006fd0 tx=0x7fb4c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:15.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.594+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4d0013070 con 0x7fb4dc076a70 2024-04-25T14:54:15.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.829+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb4dc02e090 con 0x7fb4c40396b0 2024-04-25T14:54:15.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.838+0000 7fb4da000700 1 -- 172.21.15.19:0/31020636 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4dc02e090 con 0x7fb4c40396b0 2024-04-25T14:54:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 msgr2=0x7fb4c403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 0x7fb4c403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8006fd0 tx=0x7fb4c8006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 msgr2=0x7fb4dc19df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005950 tx=0x7fb4d0004fa0 comp rx=0 tx=0).stop 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 shutdown_connections 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c40396b0 0x7fb4c403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 --2- 172.21.15.19:0/31020636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc076a70 0x7fb4dc19df60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 >> 172.21.15.19:0/31020636 conn(0x7fb4dc0fd8f0 msgr2=0x7fb4dc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 shutdown_connections 2024-04-25T14:54:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:15.841+0000 7fb4e2800700 1 -- 172.21.15.19:0/31020636 wait complete. 2024-04-25T14:54:16.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:16.638 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi083 2024-04-25T14:54:16.639 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:54:16.639 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:54:16.657 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:54:16.657 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:16.714 INFO:tasks.cephadm:Adding host smithi083 to orchestrator... 2024-04-25T14:54:16.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch host add smithi083 2024-04-25T14:54:16.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:16.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:16.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:17.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T14:54:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:16 smithi019 ceph-mon[28090]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:17.146 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:18.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:17 smithi019 ceph-mon[28090]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:19.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 -- 172.21.15.19:0/1318405247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 msgr2=0x7f3e781088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:19.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/1318405247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e781088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009910 tx=0x7f3e60009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 -- 172.21.15.19:0/1318405247 shutdown_connections 2024-04-25T14:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/1318405247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e781088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 -- 172.21.15.19:0/1318405247 >> 172.21.15.19:0/1318405247 conn(0x7f3e780fdf80 msgr2=0x7f3e781003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 -- 172.21.15.19:0/1318405247 shutdown_connections 2024-04-25T14:54:19.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.197+0000 7f3e7fa00700 1 -- 172.21.15.19:0/1318405247 wait complete. 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.198+0000 7f3e7fa00700 1 Processor -- start 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7fa00700 1 -- start start 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7fa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7fa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810c940 con 0x7f3e781084e0 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60514/0 (socket says 172.21.15.19:60514) 2024-04-25T14:54:19.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.199+0000 7f3e7d000700 1 -- 172.21.15.19:0/378475529 learned_addr learned my addr 172.21.15.19:0/378475529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e7d000700 1 -- 172.21.15.19:0/378475529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7819a5f0 con 0x7f3e781084e0 2024-04-25T14:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e7d000700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e60026070 con 0x7f3e781084e0 2024-04-25T14:54:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e6000bba0 con 0x7f3e781084e0 2024-04-25T14:54:19.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e60003910 con 0x7f3e781084e0 2024-04-25T14:54:19.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.200+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7819a8e0 con 0x7f3e781084e0 2024-04-25T14:54:19.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.201+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7819e980 con 0x7f3e781084e0 2024-04-25T14:54:19.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.202+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e78103b90 con 0x7f3e781084e0 2024-04-25T14:54:19.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.202+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3e60029070 con 0x7f3e781084e0 2024-04-25T14:54:19.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.202+0000 7f3e76000700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 0x7f3e5803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:19.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.203+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e6001e070 con 0x7f3e781084e0 2024-04-25T14:54:19.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.203+0000 7f3e6fe00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 0x7f3e5803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:19.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.204+0000 7f3e6fe00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 0x7f3e5803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:19.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.208+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e60013070 con 0x7f3e781084e0 2024-04-25T14:54:19.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:19.442+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}) -- 0x7f3e7819b410 con 0x7f3e580396b0 2024-04-25T14:54:19.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:19 smithi019 ceph-mon[28090]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:54:19.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:19 smithi019 ceph-mon[28090]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:19.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:19.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:19.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:20 smithi019 ceph-mon[28090]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:21.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:21 smithi019 ceph-mon[28090]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:21.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:21 smithi019 ceph-mon[28090]: Deploying cephadm binary to smithi083 2024-04-25T14:54:23.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:23 smithi019 ceph-mon[28090]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:24.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.242+0000 7f3e76000700 1 -- 172.21.15.19:0/378475529 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3e7819b410 con 0x7f3e580396b0 2024-04-25T14:54:24.246 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi083' with addr '172.21.15.83' 2024-04-25T14:54:24.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.245+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 msgr2=0x7f3e5803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:24.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.245+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 0x7f3e5803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006fd0 tx=0x7f3e68006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:24.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.245+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 msgr2=0x7f3e7819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.245+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60005780 comp rx=0 tx=0).stop 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 shutdown_connections 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3e580396b0 0x7f3e5803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 --2- 172.21.15.19:0/378475529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e781084e0 0x7f3e7819a0b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 >> 172.21.15.19:0/378475529 conn(0x7f3e780fdf80 msgr2=0x7f3e7810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 shutdown_connections 2024-04-25T14:54:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:24.246+0000 7f3e7fa00700 1 -- 172.21.15.19:0/378475529 wait complete. 2024-04-25T14:54:25.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:54:25.437 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:25.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:25 smithi019 ceph-mon[28090]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:25.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:25.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:25 smithi019 ceph-mon[28090]: Added host smithi083 2024-04-25T14:54:25.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:25.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:26.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:26 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 -- 172.21.15.19:0/78536962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 msgr2=0x7f8f5c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 --2- 172.21.15.19:0/78536962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c10dd40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009910 tx=0x7f8f4c009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 -- 172.21.15.19:0/78536962 shutdown_connections 2024-04-25T14:54:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 --2- 172.21.15.19:0/78536962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c10dd40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 -- 172.21.15.19:0/78536962 >> 172.21.15.19:0/78536962 conn(0x7f8f5c0781e0 msgr2=0x7f8f5c07a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 -- 172.21.15.19:0/78536962 shutdown_connections 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.527+0000 7f8f61200700 1 -- 172.21.15.19:0/78536962 wait complete. 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.528+0000 7f8f61200700 1 Processor -- start 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.528+0000 7f8f61200700 1 -- start start 2024-04-25T14:54:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.529+0000 7f8f61200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.529+0000 7f8f61200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c112f60 con 0x7f8f5c10d960 2024-04-25T14:54:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.529+0000 7f8f59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.529+0000 7f8f59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55980/0 (socket says 172.21.15.19:55980) 2024-04-25T14:54:27.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.529+0000 7f8f59e00700 1 -- 172.21.15.19:0/1499378398 learned_addr learned my addr 172.21.15.19:0/1499378398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.530+0000 7f8f59e00700 1 -- 172.21.15.19:0/1499378398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c19e9b0 con 0x7f8f5c10d960 2024-04-25T14:54:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.530+0000 7f8f59e00700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c006010 tx=0x7f8f4c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.531+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f4c026070 con 0x7f8f5c10d960 2024-04-25T14:54:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.531+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f4c00bba0 con 0x7f8f5c10d960 2024-04-25T14:54:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.531+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f4c003930 con 0x7f8f5c10d960 2024-04-25T14:54:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.531+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c19eca0 con 0x7f8f5c10d960 2024-04-25T14:54:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.531+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c1a2d40 con 0x7f8f5c10d960 2024-04-25T14:54:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.532+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c10a180 con 0x7f8f5c10d960 2024-04-25T14:54:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.532+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8f4c029070 con 0x7f8f5c10d960 2024-04-25T14:54:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.533+0000 7f8f52a00700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 0x7f8f4003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.533+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f4c01e070 con 0x7f8f5c10d960 2024-04-25T14:54:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.533+0000 7f8f59400700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 0x7f8f4003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.534+0000 7f8f59400700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 0x7f8f4003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8f48006fd0 tx=0x7f8f48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.539+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f4c013070 con 0x7f8f5c10d960 2024-04-25T14:54:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:27 smithi019 ceph-mon[28090]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.776+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f5c19f7d0 con 0x7f8f400396b0 2024-04-25T14:54:27.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.778+0000 7f8f52a00700 1 -- 172.21.15.19:0/1499378398 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f8f5c19f7d0 con 0x7f8f400396b0 2024-04-25T14:54:27.780 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:27.780 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}] 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.781+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 msgr2=0x7f8f4003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.781+0000 7f8f61200700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 0x7f8f4003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8f48006fd0 tx=0x7f8f48006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.781+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 msgr2=0x7f8f5c19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.781+0000 7f8f61200700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c006010 tx=0x7f8f4c005780 comp rx=0 tx=0).stop 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 shutdown_connections 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f8f400396b0 0x7f8f4003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 --2- 172.21.15.19:0/1499378398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5c10d960 0x7f8f5c19e470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 >> 172.21.15.19:0/1499378398 conn(0x7f8f5c0781e0 msgr2=0x7f8f5c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 shutdown_connections 2024-04-25T14:54:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:27.782+0000 7f8f61200700 1 -- 172.21.15.19:0/1499378398 wait complete. 2024-04-25T14:54:28.470 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-04-25T14:54:28.470 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:54:28.470 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:54:28.487 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:54:28.488 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:28.544 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-04-25T14:54:28.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch host add smithi102 2024-04-25T14:54:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:29.440 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:29 smithi019 ceph-mon[28090]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:54:29.440 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:29 smithi019 ceph-mon[28090]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 -- 172.21.15.19:0/2987837332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 msgr2=0x7fcea0075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 --2- 172.21.15.19:0/2987837332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea0075250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce90009910 tx=0x7fce90009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 -- 172.21.15.19:0/2987837332 shutdown_connections 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 --2- 172.21.15.19:0/2987837332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea0075250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 -- 172.21.15.19:0/2987837332 >> 172.21.15.19:0/2987837332 conn(0x7fcea00fdf80 msgr2=0x7fcea01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.205+0000 7fcea5a00700 1 -- 172.21.15.19:0/2987837332 shutdown_connections 2024-04-25T14:54:31.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.206+0000 7fcea5a00700 1 -- 172.21.15.19:0/2987837332 wait complete. 2024-04-25T14:54:31.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fcea5a00700 1 Processor -- start 2024-04-25T14:54:31.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fcea5a00700 1 -- start start 2024-04-25T14:54:31.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fcea5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:31.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fcea5a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcea010ebb0 con 0x7fcea0074e70 2024-04-25T14:54:31.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:31.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.207+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55990/0 (socket says 172.21.15.19:55990) 2024-04-25T14:54:31.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.208+0000 7fce9ea00700 1 -- 172.21.15.19:0/1289671133 learned_addr learned my addr 172.21.15.19:0/1289671133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:31.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.208+0000 7fce9ea00700 1 -- 172.21.15.19:0/1289671133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea019a640 con 0x7fcea0074e70 2024-04-25T14:54:31.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.208+0000 7fce9ea00700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.208+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90026070 con 0x7fcea0074e70 2024-04-25T14:54:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.209+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9000bc00 con 0x7fcea0074e70 2024-04-25T14:54:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.209+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea019a8d0 con 0x7fcea0074e70 2024-04-25T14:54:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.209+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90003910 con 0x7fcea0074e70 2024-04-25T14:54:31.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.209+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea019ad30 con 0x7fcea0074e70 2024-04-25T14:54:31.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.210+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fce90029070 con 0x7fcea0074e70 2024-04-25T14:54:31.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.210+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea0076520 con 0x7fcea0074e70 2024-04-25T14:54:31.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.210+0000 7fce97400700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 0x7fce8403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:31.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.210+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce9001e070 con 0x7fcea0074e70 2024-04-25T14:54:31.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.211+0000 7fce9e000700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 0x7fce8403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:31.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.211+0000 7fce9e000700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 0x7fce8403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006fd0 tx=0x7fce8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:31.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.216+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce90013070 con 0x7fcea0074e70 2024-04-25T14:54:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:31.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:31.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:31.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:31.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:54:31.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:31 smithi019 ceph-mon[28090]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:31.457+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) -- 0x7fcea019b4d0 con 0x7fce840396b0 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:32 smithi019 ceph-mon[28090]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:33.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:33 smithi019 ceph-mon[28090]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:33.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:33 smithi019 ceph-mon[28090]: Deploying cephadm binary to smithi102 2024-04-25T14:54:35.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:35 smithi019 ceph-mon[28090]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.319+0000 7fce97400700 1 -- 172.21.15.19:0/1289671133 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcea019b4d0 con 0x7fce840396b0 2024-04-25T14:54:36.321 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-04-25T14:54:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 msgr2=0x7fce8403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 0x7fce8403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006fd0 tx=0x7fce8c006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 msgr2=0x7fcea019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90005780 comp rx=0 tx=0).stop 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 shutdown_connections 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fce840396b0 0x7fce8403bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 --2- 172.21.15.19:0/1289671133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea0074e70 0x7fcea019a100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 >> 172.21.15.19:0/1289671133 conn(0x7fcea00fdf80 msgr2=0x7fcea01066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.323+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 shutdown_connections 2024-04-25T14:54:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:36.324+0000 7fcea5a00700 1 -- 172.21.15.19:0/1289671133 wait complete. 2024-04-25T14:54:37.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:54:37.582 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:37 smithi019 ceph-mon[28090]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:37 smithi019 ceph-mon[28090]: Added host smithi102 2024-04-25T14:54:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:38.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:38 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.727+0000 7fa5e0c00700 1 -- 172.21.15.19:0/993625212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 msgr2=0x7fa5dc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.727+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/993625212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc10dd40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009910 tx=0x7fa5cc009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.727+0000 7fa5e0c00700 1 -- 172.21.15.19:0/993625212 shutdown_connections 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.727+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/993625212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc10dd40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.727+0000 7fa5e0c00700 1 -- 172.21.15.19:0/993625212 >> 172.21.15.19:0/993625212 conn(0x7fa5dc0781e0 msgr2=0x7fa5dc07a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.728+0000 7fa5e0c00700 1 -- 172.21.15.19:0/993625212 shutdown_connections 2024-04-25T14:54:39.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.728+0000 7fa5e0c00700 1 -- 172.21.15.19:0/993625212 wait complete. 2024-04-25T14:54:39.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:39 smithi019 ceph-mon[28090]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:39.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.729+0000 7fa5e0c00700 1 Processor -- start 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.729+0000 7fa5e0c00700 1 -- start start 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.729+0000 7fa5e0c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.729+0000 7fa5e0c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc112f60 con 0x7fa5dc10d960 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.730+0000 7fa5d9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.730+0000 7fa5d9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38448/0 (socket says 172.21.15.19:38448) 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.730+0000 7fa5d9e00700 1 -- 172.21.15.19:0/258317305 learned_addr learned my addr 172.21.15.19:0/258317305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.730+0000 7fa5d9e00700 1 -- 172.21.15.19:0/258317305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19e9b0 con 0x7fa5dc10d960 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.730+0000 7fa5d9e00700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006010 tx=0x7fa5cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.731+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc026070 con 0x7fa5dc10d960 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.731+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5cc00bba0 con 0x7fa5dc10d960 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.731+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc003930 con 0x7fa5dc10d960 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.731+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19eca0 con 0x7fa5dc10d960 2024-04-25T14:54:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.731+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1a2d40 con 0x7fa5dc10d960 2024-04-25T14:54:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.732+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa5cc029070 con 0x7fa5dc10d960 2024-04-25T14:54:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.732+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc10a180 con 0x7fa5dc10d960 2024-04-25T14:54:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.733+0000 7fa5d3400700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 0x7fa5c003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.733+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa5cc01e070 con 0x7fa5dc10d960 2024-04-25T14:54:39.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.733+0000 7fa5d1400700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 0x7fa5c003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:39.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.734+0000 7fa5d1400700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 0x7fa5c003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.738+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5cc013070 con 0x7fa5dc10d960 2024-04-25T14:54:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.976+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5dc19f7d0 con 0x7fa5c00396b0 2024-04-25T14:54:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.977+0000 7fa5d3400700 1 -- 172.21.15.19:0/258317305 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fa5dc19f7d0 con 0x7fa5c00396b0 2024-04-25T14:54:39.979 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T14:54:39.979 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.981+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 msgr2=0x7fa5c003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.981+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 0x7fa5c003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.981+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 msgr2=0x7fa5dc19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.981+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006010 tx=0x7fa5cc005780 comp rx=0 tx=0).stop 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 shutdown_connections 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5c00396b0 0x7fa5c003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 --2- 172.21.15.19:0/258317305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5dc10d960 0x7fa5dc19e470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 >> 172.21.15.19:0/258317305 conn(0x7fa5dc0781e0 msgr2=0x7fa5dc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 shutdown_connections 2024-04-25T14:54:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:39.982+0000 7fa5e0c00700 1 -- 172.21.15.19:0/258317305 wait complete. 2024-04-25T14:54:40.778 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T14:54:40.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T14:54:41.242 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:41.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:41 smithi019 ceph-mon[28090]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:41.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:41 smithi019 ceph-mon[28090]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 -- 172.21.15.19:0/826877944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 msgr2=0x7fa9f810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 --2- 172.21.15.19:0/826877944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f810aa50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009910 tx=0x7fa9e8009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 -- 172.21.15.19:0/826877944 shutdown_connections 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 --2- 172.21.15.19:0/826877944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f810aa50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 -- 172.21.15.19:0/826877944 >> 172.21.15.19:0/826877944 conn(0x7fa9f8100160 msgr2=0x7fa9f8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 -- 172.21.15.19:0/826877944 shutdown_connections 2024-04-25T14:54:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.379+0000 7fa9ff000700 1 -- 172.21.15.19:0/826877944 wait complete. 2024-04-25T14:54:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.380+0000 7fa9ff000700 1 Processor -- start 2024-04-25T14:54:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.380+0000 7fa9ff000700 1 -- start start 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9ff000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9ff000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f810eb60 con 0x7fa9f810a670 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9f7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9f7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38470/0 (socket says 172.21.15.19:38470) 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9f7e00700 1 -- 172.21.15.19:0/3728103394 learned_addr learned my addr 172.21.15.19:0/3728103394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.381+0000 7fa9f7e00700 1 -- 172.21.15.19:0/3728103394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f819a600 con 0x7fa9f810a670 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9f7e00700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9e8026070 con 0x7fa9f810a670 2024-04-25T14:54:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f819a8f0 con 0x7fa9f810a670 2024-04-25T14:54:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e800bba0 con 0x7fa9f810a670 2024-04-25T14:54:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9e8003910 con 0x7fa9f810a670 2024-04-25T14:54:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.382+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f819e990 con 0x7fa9f810a670 2024-04-25T14:54:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.383+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f8105db0 con 0x7fa9f810a670 2024-04-25T14:54:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.384+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa9e8029070 con 0x7fa9f810a670 2024-04-25T14:54:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.384+0000 7fa9f4c00700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 0x7fa9dc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.384+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9e801e070 con 0x7fa9f810a670 2024-04-25T14:54:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.384+0000 7fa9f7400700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 0x7fa9dc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:43.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.385+0000 7fa9f7400700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 0x7fa9dc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006fd0 tx=0x7fa9e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.390+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9e8013070 con 0x7fa9f810a670 2024-04-25T14:54:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.628+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa9f819b420 con 0x7fa9f810a670 2024-04-25T14:54:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.684+0000 7fa9f4c00700 1 -- 172.21.15.19:0/3728103394 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa9e80282c0 con 0x7fa9f810a670 2024-04-25T14:54:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 msgr2=0x7fa9dc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 0x7fa9dc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006fd0 tx=0x7fa9e4006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 msgr2=0x7fa9f819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8005780 comp rx=0 tx=0).stop 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 shutdown_connections 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9dc0396b0 0x7fa9dc03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 --2- 172.21.15.19:0/3728103394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f810a670 0x7fa9f819a0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.688+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 >> 172.21.15.19:0/3728103394 conn(0x7fa9f8100160 msgr2=0x7fa9f8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.689+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 shutdown_connections 2024-04-25T14:54:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:54:43.689+0000 7fa9ff000700 1 -- 172.21.15.19:0/3728103394 wait complete. 2024-04-25T14:54:43.691 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:44.059 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:54:44.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:43 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:54:44.464 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-04-25T14:54:44.464 INFO:tasks.cephadm:Adding mon.b on smithi083 2024-04-25T14:54:44.464 INFO:tasks.cephadm:Adding mon.c on smithi102 2024-04-25T14:54:44.464 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c' 2024-04-25T14:54:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:54:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:54:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:54:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:46.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:45 smithi019 ceph-mon[28090]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 -- 172.21.15.102:0/1693223718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 msgr2=0x7fcf3c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1693223718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c1088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b0a0 tx=0x7fcf2c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 -- 172.21.15.102:0/1693223718 shutdown_connections 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1693223718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c1088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 -- 172.21.15.102:0/1693223718 >> 172.21.15.102:0/1693223718 conn(0x7fcf3c0fdf80 msgr2=0x7fcf3c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 -- 172.21.15.102:0/1693223718 shutdown_connections 2024-04-25T14:54:46.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.157+0000 7fcf433b8700 1 -- 172.21.15.102:0/1693223718 wait complete. 2024-04-25T14:54:46.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf433b8700 1 Processor -- start 2024-04-25T14:54:46.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf433b8700 1 -- start start 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf433b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf433b8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10c940 con 0x7fcf3c1084e0 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf41154700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf41154700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.102:57058/0 (socket says 172.21.15.102:57058) 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.158+0000 7fcf41154700 1 -- 172.21.15.102:0/1499855905 learned_addr learned my addr 172.21.15.102:0/1499855905 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:54:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.159+0000 7fcf41154700 1 -- 172.21.15.102:0/1499855905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c198370 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.159+0000 7fcf41154700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006130 tx=0x7fcf2c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.160+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf2c021020 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.160+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c198600 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.160+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c19c700 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.160+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2c009360 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.160+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf2c017040 con 0x7fcf3c1084e0 2024-04-25T14:54:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.161+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf28005330 con 0x7fcf3c1084e0 2024-04-25T14:54:46.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.161+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcf2c024020 con 0x7fcf3c1084e0 2024-04-25T14:54:46.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.161+0000 7fcf3a7fc700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 0x7fcf2403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:46.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.161+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcf2c00f050 con 0x7fcf3c1084e0 2024-04-25T14:54:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.162+0000 7fcf40953700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 0x7fcf2403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.163+0000 7fcf40953700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 0x7fcf2403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcf30006fd0 tx=0x7fcf30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:46.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.176+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf2c02b030 con 0x7fcf3c1084e0 2024-04-25T14:54:46.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.384+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}) -- 0x7fcf28000c80 con 0x7fcf24039650 2024-04-25T14:54:46.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.400+0000 7fcf3a7fc700 1 -- 172.21.15.102:0/1499855905 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcf28000c80 con 0x7fcf24039650 2024-04-25T14:54:46.402 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mon update... 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 msgr2=0x7fcf2403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 0x7fcf2403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcf30006fd0 tx=0x7fcf30006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 msgr2=0x7fcf3c197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006130 tx=0x7fcf2c0097d0 comp rx=0 tx=0).stop 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 shutdown_connections 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf24039650 0x7fcf2403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 --2- 172.21.15.102:0/1499855905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf3c1084e0 0x7fcf3c197e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 >> 172.21.15.102:0/1499855905 conn(0x7fcf3c0fdf80 msgr2=0x7fcf3c109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:46.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 shutdown_connections 2024-04-25T14:54:46.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:46.402+0000 7fcf433b8700 1 -- 172.21.15.102:0/1499855905 wait complete. 2024-04-25T14:54:47.011 DEBUG:teuthology.orchestra.run.smithi083:mon.b> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T14:54:47.013 DEBUG:teuthology.orchestra.run.smithi102:mon.c> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T14:54:47.015 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:54:47.016 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:54:47.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c;count:3 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:47.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:47 smithi019 ceph-mon[28090]: Deploying daemon mon.c on smithi102 2024-04-25T14:54:49.039 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:54:49.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:49 smithi019 ceph-mon[28090]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:50.264 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 systemd[1]: Starting Ceph mon.c for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T14:54:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 podman[32959]: 2024-04-25 14:54:50.530573823 +0000 UTC m=+0.167661447 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:54:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 podman[32959]: 2024-04-25 14:54:50.650472677 +0000 UTC m=+0.287560290 container create 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 -- 172.21.15.102:0/1895210638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 msgr2=0x7f611c101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 --2- 172.21.15.102:0/1895210638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c101730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f610c009910 tx=0x7f610c009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 -- 172.21.15.102:0/1895210638 shutdown_connections 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 --2- 172.21.15.102:0/1895210638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c101730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 -- 172.21.15.102:0/1895210638 >> 172.21.15.102:0/1895210638 conn(0x7f611c0fcf90 msgr2=0x7f611c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:51.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 -- 172.21.15.102:0/1895210638 shutdown_connections 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.956+0000 7f6121b05700 1 -- 172.21.15.102:0/1895210638 wait complete. 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.957+0000 7f6121b05700 1 Processor -- start 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.957+0000 7f6121b05700 1 -- start start 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.957+0000 7f6121b05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.957+0000 7f6121b05700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f611c10ecc0 con 0x7f611c101350 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.958+0000 7f611b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.958+0000 7f611b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.102:57080/0 (socket says 172.21.15.102:57080) 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.958+0000 7f611b7fe700 1 -- 172.21.15.102:0/900578730 learned_addr learned my addr 172.21.15.102:0/900578730 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.959+0000 7f611b7fe700 1 -- 172.21.15.102:0/900578730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c19a760 con 0x7f611c101350 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.959+0000 7f611b7fe700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f610c0056d0 tx=0x7f610c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.959+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f610c026070 con 0x7f611c101350 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.959+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c19a9f0 con 0x7f611c101350 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.960+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c19ae50 con 0x7f611c101350 2024-04-25T14:54:51.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.960+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c00b750 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.960+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f610c030060 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.960+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6108005330 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.961+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f610c029070 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.961+0000 7f6118ff9700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 0x7f610403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.961+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f610c01e070 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.962+0000 7f611affd700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 0x7f610403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.963+0000 7f611affd700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 0x7f610403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6110006fd0 tx=0x7f6110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:50.977+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f610c013070 con 0x7f611c101350 2024-04-25T14:54:51.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:51.041+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f610c021ca0 con 0x7f611c101350 2024-04-25T14:54:51.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 podman[32959]: 2024-04-25 14:54:50.962107621 +0000 UTC m=+0.599195241 container init 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:54:51.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 podman[32959]: 2024-04-25 14:54:50.965755878 +0000 UTC m=+0.602843492 container start 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: pidfile_write: ignore empty --pid-file 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: load: jerasure load: lrc 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Git sha 0 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: DB SUMMARY 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: DB Session ID: 3ERU1QTIOAMMHK00OVDJ 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T14:54:51.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.env: 0x556b81300cc0 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.info_log: 0x556b83215be0 2024-04-25T14:54:51.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.statistics: (nil) 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.use_fsync: 0 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:54:51.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.db_log_dir: 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.wal_dir: 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:54:51.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.write_buffer_manager: 0x556b831a90e0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:54:51.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.unordered_write: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.row_cache: None 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.wal_filter: None 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.wal_compression: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:54:51.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:54:51.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_open_files: -1 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Compression algorithms supported: 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:54:51.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kZSTD supported: 0 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.merge_operator: 2024-04-25T14:54:51.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_filter: None 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556b83215dc0) 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_type: 0 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_index_type: 0 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_shortening: 1 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: checksum: 4 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: no_block_cache: 0 2024-04-25T14:54:51.164 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache: 0x556b831db1f0 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_options: 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: capacity : 536870912 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_shard_bits : 4 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: strict_capacity_limit : 0 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_compressed: (nil) 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: persistent_cache: (nil) 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size: 4096 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size_deviation: 10 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_restart_interval: 16 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_block_restart_interval: 1 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: metadata_block_size: 4096 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: partition_filters: 0 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: use_delta_encoding: 1 2024-04-25T14:54:51.165 INFO:journalctl@ceph.mon.c.smithi102.stdout: filter_policy: bloomfilter 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: whole_key_filtering: 1 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: verify_compression: 0 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: format_version: 5 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_index_compression: 1 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_align: 0 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: max_auto_readahead_size: 262144 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: prepopulate_block_cache: 0 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:54:51.166 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:54:51.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:54:51.170 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression: NoCompression 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.num_levels: 7 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:54:51.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:54:51.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:54:51.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:54:51.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.ttl: 2592000 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:54:51.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:50 smithi102 ceph-mon[33005]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7be7f39a-4c4e-47d4-926b-2f4c3c1e5b1d 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056891005976, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056891006397, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714056891, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7be7f39a-4c4e-47d4-926b-2f4c3c1e5b1d", "db_session_id": "3ERU1QTIOAMMHK00OVDJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056891006489, "job": 1, "event": "recovery_finished"} 2024-04-25T14:54:51.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556b832f0000 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: DB pointer 0x556b832da000 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** DB Stats ** 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.177 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:54:51.178 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache BinnedLRUCache@0x556b831db1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: using public_addr v2:172.21.15.102:0/0 -> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: starting mon.c rank -1 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(???) e0 preinit fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T14:54:51.179 INFO:journalctl@ceph.mon.c.smithi102.stdout: e1 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout: legacy client fscid: -1 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout: No filesystems configured 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:54:51.180 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Deploying cephadm binary to smithi083 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Added host smithi083 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Deploying cephadm binary to smithi102 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Added host smithi102 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:54:51.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c;count:3 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:51.185 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: Deploying daemon mon.c on smithi102 2024-04-25T14:54:51.186 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:51.186 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 ceph-mon[33005]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:54:51.186 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 bash[32959]: 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 2024-04-25T14:54:51.186 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:51 smithi102 systemd[1]: Started Ceph mon.c for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:54:51.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:51.267+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6108005620 con 0x7f611c101350 2024-04-25T14:54:56.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.070+0000 7f6118ff9700 1 -- 172.21.15.102:0/900578730 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f610c038410 con 0x7f611c101350 2024-04-25T14:54:56.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:54:56.071 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","modified":"2024-04-25T14:54:51.040126Z","created":"2024-04-25T14:52:34.247134Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 msgr2=0x7f610403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 0x7f610403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6110006fd0 tx=0x7f6110006e40 comp rx=0 tx=0).stop 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 msgr2=0x7f611c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f610c0056d0 tx=0x7f610c0057b0 comp rx=0 tx=0).stop 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 shutdown_connections 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6104039650 0x7f610403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 --2- 172.21.15.102:0/900578730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c101350 0x7f611c19a220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 >> 172.21.15.102:0/900578730 conn(0x7f611c0fcf90 msgr2=0x7f611c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 shutdown_connections 2024-04-25T14:54:56.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:56.073+0000 7f6121b05700 1 -- 172.21.15.102:0/900578730 wait complete. 2024-04-25T14:54:56.075 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-04-25T14:54:56.392 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: mon.a calling monitor election 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/900578730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: mon.c calling monitor election 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: monmap epoch 2 2024-04-25T14:54:56.393 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: last_changed 2024-04-25T14:54:51.040126+0000 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: min_mon_release 19 (squid) 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: election_strategy: 1 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: fsmap 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: mgrmap e13: x(active, since 60s) 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: overall HEALTH_OK 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:56.394 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: mon.a calling monitor election 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/900578730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: mon.c calling monitor election 2024-04-25T14:54:56.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: monmap epoch 2 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: last_changed 2024-04-25T14:54:51.040126+0000 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: min_mon_release 19 (squid) 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: election_strategy: 1 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: fsmap 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: mgrmap e13: x(active, since 60s) 2024-04-25T14:54:56.569 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: overall HEALTH_OK 2024-04-25T14:54:56.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:56.570 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:54:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: Deploying daemon mon.b on smithi083 2024-04-25T14:54:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: Deploying daemon mon.b on smithi083 2024-04-25T14:54:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:57.693 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:54:57.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:54:57.933 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:54:58.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 14:54:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T14:54:58.041+0000 7f1940e00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T14:54:59.191 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:54:59 smithi102 ceph-mon[33005]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.362+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/196511139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 msgr2=0x7f3c78103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.362+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/196511139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c78103af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009910 tx=0x7f3c68009c20 comp rx=0 tx=0).stop 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/196511139 shutdown_connections 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/196511139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c78103af0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/196511139 >> 172.21.15.102:0/196511139 conn(0x7f3c780fcf40 msgr2=0x7f3c780ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/196511139 shutdown_connections 2024-04-25T14:54:59.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/196511139 wait complete. 2024-04-25T14:54:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.363+0000 7f3c7cb8f700 1 Processor -- start 2024-04-25T14:54:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7cb8f700 1 -- start start 2024-04-25T14:54:59.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7cb8f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c781981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7cb8f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7cb8f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810cab0 con 0x7f3c78103710 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7cb8f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810c930 con 0x7f3c781986e0 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47522/0 (socket says 172.21.15.102:47522) 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 -- 172.21.15.102:0/3647629958 learned_addr learned my addr 172.21.15.102:0/3647629958 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c7659c700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c781981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 -- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 msgr2=0x7f3c7819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T14:54:59.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 -- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 msgr2=0x7f3c7819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.364+0000 7f3c75d9b700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.365+0000 7f3c7659c700 1 -- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 msgr2=0x7f3c7819ca70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.365+0000 7f3c7659c700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.365+0000 7f3c7659c700 1 -- 172.21.15.102:0/3647629958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c78198d20 con 0x7f3c78103710 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.365+0000 7f3c7659c700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c781981a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c68014000 tx=0x7f3c68005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.365+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c68005dc0 con 0x7f3c78103710 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.366+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7819cfb0 con 0x7f3c78103710 2024-04-25T14:54:59.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.366+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6800be10 con 0x7f3c78103710 2024-04-25T14:54:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.366+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7819d4b0 con 0x7f3c78103710 2024-04-25T14:54:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.366+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c68022d50 con 0x7f3c78103710 2024-04-25T14:54:59.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.366+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7806b530 con 0x7f3c78103710 2024-04-25T14:54:59.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.367+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c6802a070 con 0x7f3c78103710 2024-04-25T14:54:59.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.367+0000 7f3c677fe700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 0x7f3c6003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:59.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.367+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c6801e030 con 0x7f3c78103710 2024-04-25T14:54:59.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.368+0000 7f3c75d9b700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 0x7f3c6003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:59.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.369+0000 7f3c75d9b700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 0x7f3c6003b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009180 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:59.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.385+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c68013070 con 0x7f3c78103710 2024-04-25T14:54:59.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:54:59 smithi019 ceph-mon[28090]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:54:59.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.661+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3c7819d760 con 0x7f3c78103710 2024-04-25T14:54:59.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.662+0000 7f3c677fe700 1 -- 172.21.15.102:0/3647629958 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3c68035090 con 0x7f3c78103710 2024-04-25T14:54:59.663 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:54:59.664 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","modified":"2024-04-25T14:54:51.040126Z","created":"2024-04-25T14:52:34.247134Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:54:59.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 msgr2=0x7f3c6003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 0x7f3c6003b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009180 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 msgr2=0x7f3c781981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c781981a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c68014000 tx=0x7f3c68005030 comp rx=0 tx=0).stop 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 shutdown_connections 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c600393f0 0x7f3c6003b8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78103710 0x7f3c781981a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 --2- 172.21.15.102:0/3647629958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c781986e0 0x7f3c7819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 >> 172.21.15.102:0/3647629958 conn(0x7f3c780fcf40 msgr2=0x7f3c781064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 shutdown_connections 2024-04-25T14:54:59.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:54:59.664+0000 7f3c7cb8f700 1 -- 172.21.15.102:0/3647629958 wait complete. 2024-04-25T14:54:59.667 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-04-25T14:55:00.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:55:00.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:55:00.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:55:00.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:55:00.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:55:00.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Deploying cephadm binary to smithi083 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Added host smithi083 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Deploying cephadm binary to smithi102 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.544 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Added host smithi102 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.545 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3728103394' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.546 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c;count:3 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Deploying daemon mon.c on smithi102 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.a calling monitor election 2024-04-25T14:55:00.547 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/900578730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.c calling monitor election 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: monmap epoch 2 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: last_changed 2024-04-25T14:54:51.040126+0000 2024-04-25T14:55:00.548 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: election_strategy: 1 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: fsmap 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mgrmap e13: x(active, since 60s) 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: overall HEALTH_OK 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:00.549 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: Deploying daemon mon.b on smithi083 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3647629958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 ceph-mon[32919]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 bash[32900]: 8e289efd1121b260b47571cb6a3ceecf5c5e3587e7939d77cfadefc51415928b 2024-04-25T14:55:00.550 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:00 smithi083 systemd[1]: Started Ceph mon.b for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:55:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:00 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3647629958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:00.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3647629958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:01.245 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:55:01.245 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:55:01.484 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:55:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: mon.c calling monitor election 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: mon.a calling monitor election 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:55:05.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: monmap epoch 3 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: min_mon_release 19 (squid) 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: election_strategy: 1 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: fsmap 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: mgrmap e13: x(active, since 69s) 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: overall HEALTH_OK 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:05 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: mon.c calling monitor election 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: mon.a calling monitor election 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: monmap epoch 3 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: min_mon_release 19 (squid) 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: election_strategy: 1 2024-04-25T14:55:05.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: fsmap 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: mgrmap e13: x(active, since 69s) 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: overall HEALTH_OK 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:05 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:05.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 -- 172.21.15.102:0/1704770074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49641037b0 msgr2=0x7f494c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1704770074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49641037b0 0x7f494c005610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4958005fa0 tx=0x7f495800ffc0 comp rx=0 tx=0).stop 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 -- 172.21.15.102:0/1704770074 shutdown_connections 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1704770074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49641037b0 0x7f494c005610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 -- 172.21.15.102:0/1704770074 >> 172.21.15.102:0/1704770074 conn(0x7f49640fcf40 msgr2=0x7f49640ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 -- 172.21.15.102:0/1704770074 shutdown_connections 2024-04-25T14:55:05.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.815+0000 7f4968d3d700 1 -- 172.21.15.102:0/1704770074 wait complete. 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.816+0000 7f4968d3d700 1 Processor -- start 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.816+0000 7f4968d3d700 1 -- start start 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 0x7f496419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f496410a450 con 0x7f4964198590 2024-04-25T14:55:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f496410a2d0 con 0x7f496419ce60 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4968d3d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f496410a5d0 con 0x7f49641037b0 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51596/0 (socket says 172.21.15.102:51596) 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 -- 172.21.15.102:0/1597692147 learned_addr learned my addr 172.21.15.102:0/1597692147 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 msgr2=0x7f4964198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 msgr2=0x7f4964198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 0x7f496419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f496259c700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4962d9d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4962d9d700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 msgr2=0x7f496419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4962d9d700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 msgr2=0x7f496419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:55:05.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4962d9d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4962d9d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 msgr2=0x7f4964198050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 msgr2=0x7f496419d310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.817+0000 7f4961d9b700 1 -- 172.21.15.102:0/1597692147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641a4bd0 con 0x7f4964198590 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f4961d9b700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 0x7f496419c920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f495400b550 tx=0x7f495400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004230 con 0x7f4964198590 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641a4e00 con 0x7f4964198590 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49540043d0 con 0x7f4964198590 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954011b90 con 0x7f4964198590 2024-04-25T14:55:05.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.818+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a5330 con 0x7f4964198590 2024-04-25T14:55:05.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.819+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f49540036a0 con 0x7f4964198590 2024-04-25T14:55:05.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.819+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49640640a0 con 0x7f4964198590 2024-04-25T14:55:05.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.819+0000 7f49537fe700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 0x7f494403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:05.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.819+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f495404c4a0 con 0x7f4964198590 2024-04-25T14:55:05.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.820+0000 7f496259c700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 0x7f494403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:05.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.821+0000 7f496259c700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 0x7f494403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f495800fb20 tx=0x7f49580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:05.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:05.827+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4954010380 con 0x7f4964198590 2024-04-25T14:55:06.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.110+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f49641a5680 con 0x7f4964198590 2024-04-25T14:55:06.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.111+0000 7f49537fe700 1 -- 172.21.15.102:0/1597692147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f4954003990 con 0x7f4964198590 2024-04-25T14:55:06.112 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-25T14:55:06.112 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","modified":"2024-04-25T14:55:00.281844Z","created":"2024-04-25T14:52:34.247134Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 msgr2=0x7f494403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 0x7f494403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f495800fb20 tx=0x7f49580058e0 comp rx=0 tx=0).stop 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 msgr2=0x7f496419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 0x7f496419c920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f495400b550 tx=0x7f495400b910 comp rx=0 tx=0).stop 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 shutdown_connections 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496419ce60 0x7f496419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49641037b0 0x7f4964198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:06.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4944039ae0 0x7f494403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:06.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 --2- 172.21.15.102:0/1597692147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4964198590 0x7f496419c920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:06.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 >> 172.21.15.102:0/1597692147 conn(0x7f49640fcf40 msgr2=0x7f49640fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:06.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 shutdown_connections 2024-04-25T14:55:06.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:06.115+0000 7f4968d3d700 1 -- 172.21.15.102:0/1597692147 wait complete. 2024-04-25T14:55:06.117 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 3 2024-04-25T14:55:06.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:06 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:06.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:06 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:06.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:06 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/1597692147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:06.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:06 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:06.675 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T14:55:06.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T14:55:06.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:06 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:06.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:06 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:06.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:06 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/1597692147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:06.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:06 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.107 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:55:07.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.c calling monitor election 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.a calling monitor election 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:55:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: monmap epoch 3 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: election_strategy: 1 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: fsmap 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mgrmap e13: x(active, since 69s) 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: overall HEALTH_OK 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:07.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:07.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:07.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:07.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:07.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/1597692147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:55:07.543 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:08.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mon.b calling monitor election 2024-04-25T14:55:08.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mon.b calling monitor election 2024-04-25T14:55:08.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mon.c calling monitor election 2024-04-25T14:55:08.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mon.a calling monitor election 2024-04-25T14:55:08.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: monmap epoch 3 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: min_mon_release 19 (squid) 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: election_strategy: 1 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: fsmap 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: mgrmap e13: x(active, since 71s) 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: overall HEALTH_OK 2024-04-25T14:55:08.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:55:08.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:07 smithi019 ceph-mon[28090]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:55:08.208 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mon.b calling monitor election 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mon.b calling monitor election 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mon.c calling monitor election 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mon.a calling monitor election 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: monmap epoch 3 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: min_mon_release 19 (squid) 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: election_strategy: 1 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:08.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: fsmap 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: mgrmap e13: x(active, since 71s) 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: overall HEALTH_OK 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:55:08.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:07 smithi102 ceph-mon[33005]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:55:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.b calling monitor election 2024-04-25T14:55:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.b calling monitor election 2024-04-25T14:55:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.c calling monitor election 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.a calling monitor election 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: monmap epoch 3 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: last_changed 2024-04-25T14:55:00.281844+0000 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: created 2024-04-25T14:52:34.247134+0000 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: election_strategy: 1 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: fsmap 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: mgrmap e13: x(active, since 71s) 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: overall HEALTH_OK 2024-04-25T14:55:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T14:55:08.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:07 smithi083 ceph-mon[32919]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.224 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.225 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.393+0000 7faf0f000700 1 -- 172.21.15.19:0/323536367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf08074de0 msgr2=0x7faf080751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.393+0000 7faf0f000700 1 --2- 172.21.15.19:0/323536367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf08074de0 0x7faf080751c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faef8009910 tx=0x7faef8009c20 comp rx=0 tx=0).stop 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.394+0000 7faf0f000700 1 -- 172.21.15.19:0/323536367 shutdown_connections 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.394+0000 7faf0f000700 1 --2- 172.21.15.19:0/323536367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf08074de0 0x7faf080751c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.394+0000 7faf0f000700 1 -- 172.21.15.19:0/323536367 >> 172.21.15.19:0/323536367 conn(0x7faf080fdf80 msgr2=0x7faf081003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.394+0000 7faf0f000700 1 -- 172.21.15.19:0/323536367 shutdown_connections 2024-04-25T14:55:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.394+0000 7faf0f000700 1 -- 172.21.15.19:0/323536367 wait complete. 2024-04-25T14:55:09.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.395+0000 7faf0f000700 1 Processor -- start 2024-04-25T14:55:09.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.395+0000 7faf0f000700 1 -- start start 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf081a4960 0x7faf0819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 0x7faf0819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf08112fa0 con 0x7faf0819eb20 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf08112e20 con 0x7faf08074de0 2024-04-25T14:55:09.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0f000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faf08113120 con 0x7faf081a4960 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf07e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf07400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf081a4960 0x7faf0819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf07e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40022/0 (socket says 172.21.15.19:40022) 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf07e00700 1 -- 172.21.15.19:0/4255713561 learned_addr learned my addr 172.21.15.19:0/4255713561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.396+0000 7faf0d000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 0x7faf0819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 -- 172.21.15.19:0/4255713561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf081a4960 msgr2=0x7faf0819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf081a4960 0x7faf0819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 -- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 msgr2=0x7faf0819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 0x7faf0819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 -- 172.21.15.19:0/4255713561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081ab2f0 con 0x7faf08074de0 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf0d000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 0x7faf0819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf07e00700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faef8004f80 tx=0x7faef8004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8026070 con 0x7faf08074de0 2024-04-25T14:55:09.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.397+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf081ab4c0 con 0x7faf08074de0 2024-04-25T14:55:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.398+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef800b750 con 0x7faf08074de0 2024-04-25T14:55:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.398+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8003950 con 0x7faf08074de0 2024-04-25T14:55:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.398+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf081ab9f0 con 0x7faf08074de0 2024-04-25T14:55:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.399+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf080640a0 con 0x7faf08074de0 2024-04-25T14:55:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.399+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faef8029070 con 0x7faf08074de0 2024-04-25T14:55:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.400+0000 7faf04c00700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 0x7faeec03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.400+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faef804d6c0 con 0x7faf08074de0 2024-04-25T14:55:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.400+0000 7faf07400700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 0x7faeec03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:09.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.401+0000 7faf07400700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 0x7faeec03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faef4009ad0 tx=0x7faef4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.405 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.405+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef8013070 con 0x7faf08074de0 2024-04-25T14:55:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.639+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7faf081abd10 con 0x7faf08074de0 2024-04-25T14:55:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.640+0000 7faf04c00700 1 -- 172.21.15.19:0/4255713561 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7faef8030020 con 0x7faf08074de0 2024-04-25T14:55:09.641 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:09.641 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T14:55:09.642 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T14:55:09.642 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 msgr2=0x7faeec03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 0x7faeec03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faef4009ad0 tx=0x7faef4009210 comp rx=0 tx=0).stop 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 msgr2=0x7faf081a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faef8004f80 tx=0x7faef8004fb0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 shutdown_connections 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7faeec039ae0 0x7faeec03bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0819eb20 0x7faf0819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08074de0 0x7faf081a4420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 --2- 172.21.15.19:0/4255713561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf081a4960 0x7faf0819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 >> 172.21.15.19:0/4255713561 conn(0x7faf080fdf80 msgr2=0x7faf080fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 shutdown_connections 2024-04-25T14:55:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:09.644+0000 7faf0f000700 1 -- 172.21.15.19:0/4255713561 wait complete. 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: Reconfiguring daemon mon.a on smithi019 2024-04-25T14:55:10.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:10 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4255713561' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.399 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T14:55:10.399 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:55:10.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:55:10.481 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:55:10.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:10.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: Reconfiguring daemon mon.a on smithi019 2024-04-25T14:55:10.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:10 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4255713561' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: Reconfiguring daemon mon.a on smithi019 2024-04-25T14:55:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:10 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4255713561' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:10.609 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:55:10.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:55:10.641 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:55:10.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:55:10.708 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:55:10.708 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:55:10.739 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:55:10.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:55:10.806 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-04-25T14:55:10.807 INFO:tasks.cephadm:Adding mgr.y on smithi083 2024-04-25T14:55:10.807 INFO:tasks.cephadm:Adding mgr.z on smithi102 2024-04-25T14:55:10.807 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi083=y;smithi102=z' 2024-04-25T14:55:11.083 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:55:11.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:11 smithi102 ceph-mon[33005]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:11.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:11 smithi083 ceph-mon[32919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:11.604 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:11 smithi019 ceph-mon[28090]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:12.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.479+0000 7fcb147fb700 1 -- 172.21.15.102:0/259082894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c1014a0 msgr2=0x7fcb0c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:12.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 --2- 172.21.15.102:0/259082894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c1018c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800b0a0 tx=0x7fcb0800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- 172.21.15.102:0/259082894 shutdown_connections 2024-04-25T14:55:12.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 --2- 172.21.15.102:0/259082894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c1018c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- 172.21.15.102:0/259082894 >> 172.21.15.102:0/259082894 conn(0x7fcb0c0fca40 msgr2=0x7fcb0c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- 172.21.15.102:0/259082894 shutdown_connections 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- 172.21.15.102:0/259082894 wait complete. 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 Processor -- start 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- start start 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c198280 0x7fcb0c19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c107390 con 0x7fcb0c198280 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c107210 con 0x7fcb0c1014a0 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.480+0000 7fcb147fb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c107510 con 0x7fcb0c198700 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51610/0 (socket says 172.21.15.102:51610) 2024-04-25T14:55:12.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 -- 172.21.15.102:0/2233465507 learned_addr learned my addr 172.21.15.102:0/2233465507 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12597700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 -- 172.21.15.102:0/2233465507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 msgr2=0x7fcb0c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 -- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c198280 msgr2=0x7fcb0c19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c198280 0x7fcb0c19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 -- 172.21.15.102:0/2233465507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a0160 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12597700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb12d98700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400b550 tx=0x7fcb0400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04015020 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb04011ce0 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a0450 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04010db0 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.481+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1a0990 con 0x7fcb0c198700 2024-04-25T14:55:12.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.482+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcb04018020 con 0x7fcb0c198700 2024-04-25T14:55:12.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.482+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c0640a0 con 0x7fcb0c198700 2024-04-25T14:55:12.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.482+0000 7fcb037fe700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 0x7fcaf803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:12.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.482+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb0404c2f0 con 0x7fcb0c198700 2024-04-25T14:55:12.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.483+0000 7fcb12597700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 0x7fcaf803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:12.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.484+0000 7fcb12597700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 0x7fcaf803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb080062a0 tx=0x7fcb08006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:12.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.488+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb0401f020 con 0x7fcb0c198700 2024-04-25T14:55:12.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.696+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi083=y;smithi102=z", "target": ["mon-mgr", ""]}) -- 0x7fcb0c062f10 con 0x7fcaf8039ae0 2024-04-25T14:55:12.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.710+0000 7fcb037fe700 1 -- 172.21.15.102:0/2233465507 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcb0c062f10 con 0x7fcaf8039ae0 2024-04-25T14:55:12.711 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mgr update... 2024-04-25T14:55:12.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.712+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 msgr2=0x7fcaf803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:12.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.712+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 0x7fcaf803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb080062a0 tx=0x7fcb08006210 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.712+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 msgr2=0x7fcb0c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.712+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400b550 tx=0x7fcb0400b860 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 shutdown_connections 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb0c198700 0x7fcb0c19f860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0c1014a0 0x7fcb0c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcaf8039ae0 0x7fcaf803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 --2- 172.21.15.102:0/2233465507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb0c198280 0x7fcb0c19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 >> 172.21.15.102:0/2233465507 conn(0x7fcb0c0fca40 msgr2=0x7fcb0c18ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 shutdown_connections 2024-04-25T14:55:12.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:55:12.713+0000 7fcb147fb700 1 -- 172.21.15.102:0/2233465507 wait complete. 2024-04-25T14:55:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: Reconfiguring daemon mon.b on smithi083 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi083=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: Saving service mgr spec with placement smithi019=x;smithi083=y;smithi102=z;count:3 2024-04-25T14:55:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: Reconfiguring daemon mon.b on smithi083 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi083=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: Saving service mgr spec with placement smithi019=x;smithi083=y;smithi102=z;count:3 2024-04-25T14:55:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: Reconfiguring daemon mon.b on smithi083 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi083=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: Saving service mgr spec with placement smithi019=x;smithi083=y;smithi102=z;count:3 2024-04-25T14:55:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:13.341 DEBUG:teuthology.orchestra.run.smithi083:mgr.y> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T14:55:13.343 DEBUG:teuthology.orchestra.run.smithi102:mgr.z> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T14:55:13.346 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T14:55:13.346 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T14:55:13.346 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:55:13.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:55:13.407 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 14:54:12.622873367 +0000 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 14:48:39.334745830 +0000 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 14:48:39.334745830 +0000 2024-04-25T14:55:13.464 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 14:48:39.334745830 +0000 2024-04-25T14:55:13.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:55:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T14:55:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T14:55:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000417435 s, 1.2 MB/s 2024-04-25T14:55:13.555 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:55:13.613 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 14:54:12.696872631 +0000 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 14:48:39.958739866 +0000 2024-04-25T14:55:13.668 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 14:48:39.958739866 +0000 2024-04-25T14:55:13.669 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 14:48:39.958739866 +0000 2024-04-25T14:55:13.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:55:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T14:55:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T14:55:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000347159 s, 1.5 MB/s 2024-04-25T14:55:13.744 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:55:13.801 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 14:54:12.759872004 +0000 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 14:48:40.575733969 +0000 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 14:48:40.575733969 +0000 2024-04-25T14:55:13.857 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 14:48:40.575733969 +0000 2024-04-25T14:55:13.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:55:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T14:55:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T14:55:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000463835 s, 1.1 MB/s 2024-04-25T14:55:13.944 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:55:14.002 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 14:54:12.827871327 +0000 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 14:48:41.236727652 +0000 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 14:48:41.236727652 +0000 2024-04-25T14:55:14.056 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 14:48:41.236727652 +0000 2024-04-25T14:55:14.057 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:55:14.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T14:55:14.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T14:55:14.140 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000386011 s, 1.3 MB/s 2024-04-25T14:55:14.144 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:55:14.201 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T14:55:14.201 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:55:14.219 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:55:14.219 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 14:54:59.736507889 +0000 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 14:47:23.317320878 +0000 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 14:47:23.317320878 +0000 2024-04-25T14:55:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 14:47:23.317320878 +0000 2024-04-25T14:55:14.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:55:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T14:55:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T14:55:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000228775 s, 2.2 MB/s 2024-04-25T14:55:14.345 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:55:14.403 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-25T14:55:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 14:54:59.736507889 +0000 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 14:47:23.650312404 +0000 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 14:47:23.650312404 +0000 2024-04-25T14:55:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 14:47:23.650312404 +0000 2024-04-25T14:55:14.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:55:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T14:55:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T14:55:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000280858 s, 1.8 MB/s 2024-04-25T14:55:14.525 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:14 smithi102 ceph-mon[33005]: Reconfiguring daemon mon.c on smithi102 2024-04-25T14:55:14.582 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:14 smithi019 ceph-mon[28090]: Reconfiguring daemon mon.c on smithi102 2024-04-25T14:55:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:55:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:14.640 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 14:54:59.736507889 +0000 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 14:47:23.981303980 +0000 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 14:47:23.981303980 +0000 2024-04-25T14:55:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 14:47:23.981303980 +0000 2024-04-25T14:55:14.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:14.701 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:14 smithi083 ceph-mon[32919]: Reconfiguring daemon mon.c on smithi102 2024-04-25T14:55:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T14:55:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T14:55:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000217202 s, 2.4 MB/s 2024-04-25T14:55:14.705 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:55:14.761 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-25T14:55:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:55:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 14:54:59.737507864 +0000 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 14:47:24.322295302 +0000 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 14:47:24.322295302 +0000 2024-04-25T14:55:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 14:47:24.322295302 +0000 2024-04-25T14:55:14.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:55:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T14:55:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T14:55:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000177923 s, 2.9 MB/s 2024-04-25T14:55:14.881 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:55:14.937 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-25T14:55:14.937 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:55:14.952 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:55:14.952 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-04-25T14:55:15.006 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:55:15.006 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-25 14:54:50.031182920 +0000 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-25 14:47:23.321183222 +0000 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-25 14:47:23.321183222 +0000 2024-04-25T14:55:15.007 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-25 14:47:23.321183222 +0000 2024-04-25T14:55:15.007 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:55:15.070 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-25T14:55:15.070 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-25T14:55:15.070 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000214896 s, 2.4 MB/s 2024-04-25T14:55:15.071 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:55:15.126 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-25 14:54:50.031182920 +0000 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-25 14:47:23.654175981 +0000 2024-04-25T14:55:15.181 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-25 14:47:23.654175981 +0000 2024-04-25T14:55:15.182 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-25 14:47:23.654175981 +0000 2024-04-25T14:55:15.182 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:55:15.243 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-25T14:55:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-25T14:55:15.295 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000217613 s, 2.4 MB/s 2024-04-25T14:55:15.296 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:55:15.351 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:15.406 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-25 14:54:50.032182899 +0000 2024-04-25T14:55:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-25 14:47:23.986168761 +0000 2024-04-25T14:55:15.407 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-25 14:47:23.986168761 +0000 2024-04-25T14:55:15.407 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-25 14:47:23.986168761 +0000 2024-04-25T14:55:15.407 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:55:15.469 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-25T14:55:15.469 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-25T14:55:15.469 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000174579 s, 2.9 MB/s 2024-04-25T14:55:15.471 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:55:15.527 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-25 14:54:50.032182899 +0000 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-25 14:47:24.320161497 +0000 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-25 14:47:24.320161497 +0000 2024-04-25T14:55:15.581 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-25 14:47:24.320161497 +0000 2024-04-25T14:55:15.581 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:55:15.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:15.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:15.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:15.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:15.642 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-25T14:55:15.642 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-25T14:55:15.643 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000215531 s, 2.4 MB/s 2024-04-25T14:55:15.643 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:55:15.698 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-04-25T14:55:15.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:16.141 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:55:16.501 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:16 smithi102 ceph-mon[33005]: Deploying daemon mgr.z on smithi102 2024-04-25T14:55:16.581 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:16 smithi019 ceph-mon[28090]: Deploying daemon mgr.z on smithi102 2024-04-25T14:55:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:16 smithi083 ceph-mon[32919]: Deploying daemon mgr.z on smithi102 2024-04-25T14:55:17.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:16 smithi102 systemd[1]: Starting Ceph mgr.z for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T14:55:17.422 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:17 smithi102 ceph-mon[33005]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:17.422 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 podman[34866]: 2024-04-25 14:55:17.071446352 +0000 UTC m=+0.017962629 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:55:17.423 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 podman[34866]: 2024-04-25 14:55:17.169883912 +0000 UTC m=+0.116400199 container create 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:55:17.423 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 podman[34866]: 2024-04-25 14:55:17.422592005 +0000 UTC m=+0.369108281 container init 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T14:55:17.457 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:17 smithi019 ceph-mon[28090]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:17.690 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:17 smithi083 ceph-mon[32919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 podman[34866]: 2024-04-25 14:55:17.425447528 +0000 UTC m=+0.371963813 container start 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 ceph-mgr[34885]: -- 172.21.15.102:0/1063511603 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564be81da5a0 con 0x564be81af000 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 bash[34866]: 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 systemd[1]: Started Ceph mgr.z for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:17.573+0000 7efc4057a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:55:17.761 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:17.760+0000 7efc4057a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:55:18.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:17.862+0000 7efc4057a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:55:18.557 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:18.169+0000 7efc4057a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:55:18.557 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:18.253+0000 7efc4057a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:55:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:18 smithi019 ceph-mon[28090]: Deploying daemon mgr.y on smithi083 2024-04-25T14:55:18.925 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.925 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.925 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:18.925 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:18.925 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:18.926 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:18.926 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:18.926 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:18 smithi083 ceph-mon[32919]: Deploying daemon mgr.y on smithi083 2024-04-25T14:55:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:55:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:55:19.036 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T14:55:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T14:55:19.036 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0486091 s, 216 MB/s 2024-04-25T14:55:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:18 smithi102 ceph-mon[33005]: Deploying daemon mgr.y on smithi083 2024-04-25T14:55:19.790 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:19 smithi102 ceph-mon[33005]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:19.790 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:19.453+0000 7efc4057a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:55:19.790 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:19.619+0000 7efc4057a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:55:19.790 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:19.789+0000 7efc4057a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:55:19.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:19 smithi083 ceph-mon[32919]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:19.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:19 smithi019 ceph-mon[28090]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:19.867 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-04-25T14:55:20.174 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 systemd[1]: Started Ceph mgr.y for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T14:55:20.313 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:55:20.369 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:20.074+0000 7efc4057a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:55:20.369 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:20.164+0000 7efc4057a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:55:20.369 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:20.366+0000 7efc4057a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:55:20.465 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:20.172+0000 7f0ecec82200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:55:20.465 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:20.358+0000 7f0ecec82200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:55:20.465 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:20.463+0000 7f0ecec82200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:55:20.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:20.560+0000 7efc4057a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:55:21.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:20.786+0000 7f0ecec82200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:55:21.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:20.874+0000 7f0ecec82200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:55:21.433 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:21.189+0000 7efc4057a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:21.434 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:21.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:21.497 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:21.715 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:21.619+0000 7efc4057a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:55:22.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:21.714+0000 7efc4057a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:55:22.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:21.928+0000 7efc4057a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:55:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.319+0000 7f2416000700 1 -- 172.21.15.19:0/2975297248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f241010a980 msgr2=0x7f241010ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.319+0000 7f2416000700 1 --2- 172.21.15.19:0/2975297248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f241010a980 0x7f241010ce10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f240800b0a0 tx=0x7f240800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.319+0000 7f2416000700 1 -- 172.21.15.19:0/2975297248 shutdown_connections 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.319+0000 7f2416000700 1 --2- 172.21.15.19:0/2975297248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f241010a980 0x7f241010ce10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.320+0000 7f2416000700 1 --2- 172.21.15.19:0/2975297248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410076a10 0x7f2410074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.320+0000 7f2416000700 1 --2- 172.21.15.19:0/2975297248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2410075440 0x7f24100758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.320+0000 7f2416000700 1 -- 172.21.15.19:0/2975297248 >> 172.21.15.19:0/2975297248 conn(0x7f24100fe040 msgr2=0x7f2410100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.320+0000 7f2416000700 1 -- 172.21.15.19:0/2975297248 shutdown_connections 2024-04-25T14:55:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.320+0000 7f2416000700 1 -- 172.21.15.19:0/2975297248 wait complete. 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 Processor -- start 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 -- start start 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 0x7f241019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 0x7f24101a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110740 con 0x7f2410075440 2024-04-25T14:55:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24101105c0 con 0x7f241010a980 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.321+0000 7f2416000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f24101108c0 con 0x7f2410076a10 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 0x7f241019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 0x7f241019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40304/0 (socket says 172.21.15.19:40304) 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:34274/0 (socket says 172.21.15.19:34274) 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240f400700 1 -- 172.21.15.19:0/263198121 learned_addr learned my addr 172.21.15.19:0/263198121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:55:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240fe00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 0x7f24101a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 -- 172.21.15.19:0/263198121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 msgr2=0x7f24101a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 0x7f24101a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 -- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 msgr2=0x7f241019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 0x7f241019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 -- 172.21.15.19:0/263198121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101a29f0 con 0x7f2410076a10 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240fe00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 0x7f24101a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.322+0000 7f240ea00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e9f0 tx=0x7f23fc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.323+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00cbd0 con 0x7f2410076a10 2024-04-25T14:55:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.323+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc00cd70 con 0x7f2410076a10 2024-04-25T14:55:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.323+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc018680 con 0x7f2410076a10 2024-04-25T14:55:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.323+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101a2ce0 con 0x7f2410076a10 2024-04-25T14:55:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.323+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a31c0 con 0x7f2410076a10 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.324+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f23fc018820 con 0x7f2410076a10 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.324+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241002df80 con 0x7f2410076a10 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.325+0000 7f2407e00700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 0x7f23f403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.325+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23fc04cc00 con 0x7f2410076a10 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.325+0000 7f240f400700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 0x7f23f403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.325+0000 7f240f400700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 0x7f23f403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f241019c280 tx=0x7f2400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.330+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23fc00fc70 con 0x7f2410076a10 2024-04-25T14:55:22.458 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:22.126+0000 7f0ecec82200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:55:22.458 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:22.290+0000 7f0ecec82200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:55:22.458 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:22.456+0000 7f0ecec82200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:55:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:22.574+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2410062e80 con 0x7f23f4039890 2024-04-25T14:55:22.775 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:22.446+0000 7efc4057a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:55:22.775 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:22.536+0000 7efc4057a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:55:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.027 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:22.736+0000 7f0ecec82200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:55:23.027 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:22.827+0000 7f0ecec82200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:55:23.027 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:23.025+0000 7f0ecec82200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.028 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T14:55:23.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:23.030 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:23.030 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:23.030 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:22.774+0000 7efc4057a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:22.907+0000 7efc4057a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:23.000+0000 7efc4057a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:23.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:23.290 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:23.212+0000 7f0ecec82200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:55:23.415 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:23.114+0000 7efc4057a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:55:23.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:23.415+0000 7efc4057a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:55:24.217 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:23.798+0000 7f0ecec82200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:55:24.307 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:24.009+0000 7efc4057a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:55:24.307 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T14:55:24.221+0000 7efc4057a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:55:24.508 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:24.215+0000 7f0ecec82200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:55:24.508 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:24.310+0000 7f0ecec82200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:55:24.508 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:24.506+0000 7f0ecec82200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:55:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:24.953+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f23fc0178f0 con 0x7f2410076a10 2024-04-25T14:55:25.081 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: Standby manager daemon z started 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:25.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.290 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.022+0000 7f0ecec82200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:55:25.290 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.111+0000 7f0ecec82200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:55:25.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: Standby manager daemon z started 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:25.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: Standby manager daemon z started 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/3623590783' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:25.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:25.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:25.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:25.686 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.350+0000 7f0ecec82200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:55:25.686 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.480+0000 7f0ecec82200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:55:25.686 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.572+0000 7f0ecec82200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:55:25.686 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.684+0000 7f0ecec82200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:55:26.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-mon[32919]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T14:55:26.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:55:26.291 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:25.980+0000 7f0ecec82200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:55:26.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:25 smithi102 ceph-mon[33005]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T14:55:26.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:55:26.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:25 smithi019 ceph-mon[28090]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T14:55:26.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:55:26.961 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:26.573+0000 7f0ecec82200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:55:26.962 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T14:55:26.785+0000 7f0ecec82200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:55:26.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:26.985+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mon.1 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f23fc0173d0 con 0x7f2410076a10 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: Standby manager daemon y started 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:55:27.082 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:27.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:27.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: Standby manager daemon y started 2024-04-25T14:55:27.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:55:27.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:27.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:55:27.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: Standby manager daemon y started 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:55:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/438656250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3175702182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]': finished 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:55:28.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3175702182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]': finished 2024-04-25T14:55:28.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:55:28.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3175702182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]: dispatch 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8649bfaf-7456-4969-a41c-c8752a4bf853"}]': finished 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:55:28.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:29.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:28 smithi083 ceph-mon[32919]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:29.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:28 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/174136230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:29.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:28 smithi102 ceph-mon[33005]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:29.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:28 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/174136230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:29.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:28 smithi019 ceph-mon[28090]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:29.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:28 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/174136230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:31.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:30 smithi083 ceph-mon[32919]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:31.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:30 smithi102 ceph-mon[33005]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:30 smithi019 ceph-mon[28090]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:33.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:32 smithi019 ceph-mon[28090]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:33.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:32 smithi083 ceph-mon[32919]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:33.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:32 smithi102 ceph-mon[33005]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:35.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:34 smithi019 ceph-mon[28090]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:35.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:34 smithi083 ceph-mon[32919]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:35.306 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:34 smithi102 ceph-mon[33005]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:37.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:37 smithi083 ceph-mon[32919]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:37.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:37 smithi102 ceph-mon[33005]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:37.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:37 smithi019 ceph-mon[28090]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:38.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:38 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:55:38.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:38 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:38 smithi083 ceph-mon[32919]: Deploying daemon osd.0 on smithi019 2024-04-25T14:55:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:38 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:55:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:38 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:38 smithi102 ceph-mon[33005]: Deploying daemon osd.0 on smithi019 2024-04-25T14:55:38.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:38 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:55:38.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:38 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:38 smithi019 ceph-mon[28090]: Deploying daemon osd.0 on smithi019 2024-04-25T14:55:39.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:39 smithi019 ceph-mon[28090]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:39.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:39 smithi083 ceph-mon[32919]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:39.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:39 smithi102 ceph-mon[33005]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:41.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:41 smithi019 ceph-mon[28090]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:41.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:41 smithi083 ceph-mon[32919]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:41.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:41 smithi102 ceph-mon[33005]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:43.254 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:43 smithi019 ceph-mon[28090]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:43.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:43 smithi083 ceph-mon[32919]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:43 smithi102 ceph-mon[33005]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:45 smithi019 ceph-mon[28090]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:45.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:45 smithi083 ceph-mon[32919]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:45 smithi102 ceph-mon[33005]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:47.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:47 smithi019 ceph-mon[28090]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:47.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:47.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:47 smithi083 ceph-mon[32919]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:47.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:47 smithi102 ceph-mon[33005]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:49.219 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:49.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:49.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:49.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:49.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:50.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:50 smithi019 ceph-mon[28090]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:55:50.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:50 smithi083 ceph-mon[32919]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:55:50.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:50 smithi102 ceph-mon[33005]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:55:51.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.226+0000 7f2407e00700 1 -- 172.21.15.19:0/263198121 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2410062e80 con 0x7f23f4039890 2024-04-25T14:55:51.228 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.230+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 msgr2=0x7f23f403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.230+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 0x7f23f403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f241019c280 tx=0x7f2400009260 comp rx=0 tx=0).stop 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.230+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 msgr2=0x7f241019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.230+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e9f0 tx=0x7f23fc00ed00 comp rx=0 tx=0).stop 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 shutdown_connections 2024-04-25T14:55:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241010a980 0x7f24101a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23f4039890 0x7f23f403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2410075440 0x7f241019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 --2- 172.21.15.19:0/263198121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2410076a10 0x7f241019b2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 >> 172.21.15.19:0/263198121 conn(0x7f24100fe040 msgr2=0x7f24101068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 shutdown_connections 2024-04-25T14:55:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:55:51.231+0000 7f2416000700 1 -- 172.21.15.19:0/263198121 wait complete. 2024-04-25T14:55:51.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:51 smithi019 ceph-mon[28090]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:51.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:51 smithi019 ceph-mon[28090]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:55:51.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:51 smithi019 ceph-mon[28090]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:55:51.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:51 smithi019 ceph-mon[28090]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:55:51.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:51.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:51 smithi083 ceph-mon[32919]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:51.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:51 smithi083 ceph-mon[32919]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:55:51.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:51 smithi083 ceph-mon[32919]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:55:51.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:51 smithi083 ceph-mon[32919]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:55:51.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:51.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:51 smithi102 ceph-mon[33005]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:51.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:51 smithi102 ceph-mon[33005]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:55:51.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:51 smithi102 ceph-mon[33005]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:55:51.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:51 smithi102 ceph-mon[33005]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:55:51.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:51.983 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T14:55:51.988 INFO:tasks.cephadm:Deploying osd.1 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-25T14:55:51.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:55:52.231 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:52 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:52.494 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0[42768]: 2024-04-25T14:55:52.236+0000 7fe61c000700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T14:55:52.494 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:55:52 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0[42768]: 2024-04-25T14:55:52.268+0000 7fe611e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:52 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:53 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:55:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:53 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:55:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:53 smithi019 ceph-mon[28090]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:53.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:53 smithi019 ceph-mon[28090]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' 2024-04-25T14:55:53.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:53 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:55:53.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:53 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:55:53.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:53 smithi083 ceph-mon[32919]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:53.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:53.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:53 smithi083 ceph-mon[32919]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' 2024-04-25T14:55:53.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:53 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:55:53.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:53 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:55:53.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:53 smithi102 ceph-mon[33005]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:53.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:53.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:53 smithi102 ceph-mon[33005]: from='osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993]' entity='osd.0' 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011169 s, 939 MB/s 2024-04-25T14:55:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T14:55:54.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:54 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:54 smithi019 ceph-mon[28090]: osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] boot 2024-04-25T14:55:54.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:54 smithi019 ceph-mon[28090]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:55:54.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:54 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:54 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:54 smithi083 ceph-mon[32919]: osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] boot 2024-04-25T14:55:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:54 smithi083 ceph-mon[32919]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:55:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:54 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:54 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:54 smithi102 ceph-mon[33005]: osd.0 [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] boot 2024-04-25T14:55:54.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:54 smithi102 ceph-mon[33005]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:55:54.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:54 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:55:54.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-25T14:55:55.065 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:55:55.504 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:55 smithi083 ceph-mon[32919]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:55.504 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:55 smithi083 ceph-mon[32919]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:55:55.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:55 smithi102 ceph-mon[33005]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:55.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:55 smithi102 ceph-mon[33005]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:55:55.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:55 smithi019 ceph-mon[28090]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:55:55.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:55 smithi019 ceph-mon[28090]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:55:56.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 -- 172.21.15.83:0/3560855626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c103ca0 msgr2=0x7f9b4c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 --2- 172.21.15.83:0/3560855626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c104100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b480099c0 tx=0x7f9b48009cd0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 -- 172.21.15.83:0/3560855626 shutdown_connections 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 --2- 172.21.15.83:0/3560855626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c104640 0x7f9b4c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 --2- 172.21.15.83:0/3560855626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c102aa0 0x7f9b4c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 --2- 172.21.15.83:0/3560855626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c104100 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 -- 172.21.15.83:0/3560855626 >> 172.21.15.83:0/3560855626 conn(0x7f9b4c0fe040 msgr2=0x7f9b4c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 -- 172.21.15.83:0/3560855626 shutdown_connections 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.588+0000 7f9b54023700 1 -- 172.21.15.83:0/3560855626 wait complete. 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 Processor -- start 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 -- start start 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c102aa0 0x7f9b4c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:56.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10e590 con 0x7f9b4c102aa0 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10e410 con 0x7f9b4c104640 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b54023700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10e710 con 0x7f9b4c103ca0 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b525c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b525c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38996/0 (socket says 172.21.15.83:38996) 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.589+0000 7f9b525c0700 1 -- 172.21.15.83:0/1335876515 learned_addr learned my addr 172.21.15.83:0/1335876515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b515be700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 -- 172.21.15.83:0/1335876515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 msgr2=0x7f9b4c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 -- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c102aa0 msgr2=0x7f9b4c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c102aa0 0x7f9b4c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:56.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 -- 172.21.15.83:0/1335876515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c1a08e0 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b515be700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b525c0700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b970 tx=0x7f9b4400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44019070 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b44014440 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400ce60 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c1a0bd0 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.590+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c1a10c0 con 0x7f9b4c104640 2024-04-25T14:55:56.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.591+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c069730 con 0x7f9b4c104640 2024-04-25T14:55:56.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.592+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f9b4401c070 con 0x7f9b4c104640 2024-04-25T14:55:56.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.593+0000 7f9b42ffd700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 0x7f9b380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:56.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.593+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f9b440ce260 con 0x7f9b4c104640 2024-04-25T14:55:56.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.595+0000 7f9b51dbf700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 0x7f9b380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:56.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.595+0000 7f9b51dbf700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 0x7f9b380a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c199b40 tx=0x7f9b3c009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.596+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b4409c6b0 con 0x7f9b4c104640 2024-04-25T14:55:56.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:55:56.801+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9b4c062e80 con 0x7f9b380a3bb0 2024-04-25T14:55:57.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:57.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:57.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:57.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:55:57.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:55:58.539 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:58 smithi083 ceph-mon[32919]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:58.539 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:58 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi019 2024-04-25T14:55:58.539 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:58 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 17094M 2024-04-25T14:55:58.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:58 smithi019 ceph-mon[28090]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:58.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:58 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi019 2024-04-25T14:55:58.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:58 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 17094M 2024-04-25T14:55:58.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:58 smithi102 ceph-mon[33005]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:58.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:58 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi019 2024-04-25T14:55:58.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:58 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 17094M 2024-04-25T14:55:59.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:55:59 smithi083 ceph-mon[32919]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:55:59.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:55:59 smithi102 ceph-mon[33005]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:55:59.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:55:59 smithi019 ceph-mon[28090]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/4253024659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]': finished 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:00 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/2177008058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:00.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/4253024659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]': finished 2024-04-25T14:56:00.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:56:00.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:00.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/2177008058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/4253024659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]: dispatch 2024-04-25T14:56:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5271f6-eea9-4b6f-8b71-6c3f259b237e"}]': finished 2024-04-25T14:56:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:56:00.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:00.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/2177008058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:01.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:01 smithi083 ceph-mon[32919]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:01.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:01 smithi102 ceph-mon[33005]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:01.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:01 smithi019 ceph-mon[28090]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:03.659 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:03 smithi083 ceph-mon[32919]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:03 smithi102 ceph-mon[33005]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:03.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:03 smithi019 ceph-mon[28090]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:05.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:05 smithi083 ceph-mon[32919]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:05 smithi102 ceph-mon[33005]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:05.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:05 smithi019 ceph-mon[28090]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:07.715 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:07 smithi083 ceph-mon[32919]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:07.715 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:56:07.715 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:07 smithi102 ceph-mon[33005]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:56:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:07.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:07 smithi019 ceph-mon[28090]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:07.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:56:07.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:08.763 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:08 smithi083 ceph-mon[32919]: Deploying daemon osd.1 on smithi083 2024-04-25T14:56:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:08 smithi102 ceph-mon[33005]: Deploying daemon osd.1 on smithi083 2024-04-25T14:56:08.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:08 smithi019 ceph-mon[28090]: Deploying daemon osd.1 on smithi083 2024-04-25T14:56:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:09 smithi019 ceph-mon[28090]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:09.707 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:09 smithi083 ceph-mon[32919]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:09.806 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:09 smithi102 ceph-mon[33005]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:11.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:11 smithi083 ceph-mon[32919]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:11 smithi102 ceph-mon[33005]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:11.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:11 smithi019 ceph-mon[28090]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:13.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:13 smithi083 ceph-mon[32919]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:13.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:13 smithi019 ceph-mon[28090]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:13 smithi102 ceph-mon[33005]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:15.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:15.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:15.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.892+0000 7f9b42ffd700 1 -- 172.21.15.83:0/1335876515 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b4c062e80 con 0x7f9b380a3bb0 2024-04-25T14:56:15.893 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 1 on host 'smithi083' 2024-04-25T14:56:15.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 msgr2=0x7f9b380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:15.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 0x7f9b380a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c199b40 tx=0x7f9b3c009470 comp rx=0 tx=0).stop 2024-04-25T14:56:15.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 msgr2=0x7f9b4c1a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:15.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b970 tx=0x7f9b4400bd30 comp rx=0 tx=0).stop 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 shutdown_connections 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b4c104640 0x7f9b4c1a0210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b380a3bb0 0x7f9b380a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c102aa0 0x7f9b4c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 --2- 172.21.15.83:0/1335876515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b4c103ca0 0x7f9b4c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.894+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 >> 172.21.15.83:0/1335876515 conn(0x7f9b4c0fe040 msgr2=0x7f9b4c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.895+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 shutdown_connections 2024-04-25T14:56:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:56:15.895+0000 7f9b54023700 1 -- 172.21.15.83:0/1335876515 wait complete. 2024-04-25T14:56:16.490 DEBUG:teuthology.orchestra.run.smithi083:osd.1> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T14:56:16.492 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_4... 2024-04-25T14:56:16.492 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:56:16.731 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.018 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:16 smithi083 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:16 smithi019 ceph-mon[28090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:16 smithi102 ceph-mon[33005]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:56:17.900 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:17 smithi083 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:56:17.900 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:17 smithi083 ceph-mon[32919]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:17.900 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:17 smithi083 ceph-mon[32919]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:56:17.900 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:17.900 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:17 smithi083 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:17.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:17 smithi102 ceph-mon[33005]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:56:17.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:17 smithi102 ceph-mon[33005]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:17.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:17 smithi102 ceph-mon[33005]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:56:17.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:17.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:17 smithi102 ceph-mon[33005]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:18.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:17 smithi019 ceph-mon[28090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:56:18.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:17 smithi019 ceph-mon[28090]: from='osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:18.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:17 smithi019 ceph-mon[28090]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:56:18.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:18.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:17 smithi019 ceph-mon[28090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:56:18.524 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:56:18.525 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:56:18.525 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-25T14:56:18.525 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-25T14:56:18.525 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130452 s, 804 MB/s 2024-04-25T14:56:18.525 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-25T14:56:19.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:56:19.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:56:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: from='osd.1 ' entity='osd.1' 2024-04-25T14:56:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.041 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1[37511]: 2024-04-25T14:56:18.774+0000 7f634a661700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T14:56:19.041 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1[37511]: 2024-04-25T14:56:18.950+0000 7f634244c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: from='osd.1 ' entity='osd.1' 2024-04-25T14:56:19.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: from='osd.1 ' entity='osd.1' 2024-04-25T14:56:19.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:19.392 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi083 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:20.183 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T14:56:20.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:20.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:20.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.184 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:19 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:20.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:56:20.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:56:20.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi083 2024-04-25T14:56:20.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:19 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:20.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:56:20.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:56:20.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi083 2024-04-25T14:56:20.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:20.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:19 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:20.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 -- 172.21.15.102:0/2603016859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 msgr2=0x7f6520104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 --2- 172.21.15.102:0/2603016859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520104130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514009ca0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 -- 172.21.15.102:0/2603016859 shutdown_connections 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 --2- 172.21.15.102:0/2603016859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6520104670 0x7f652010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 --2- 172.21.15.102:0/2603016859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 0x7f6520102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 --2- 172.21.15.102:0/2603016859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520104130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 -- 172.21.15.102:0/2603016859 >> 172.21.15.102:0/2603016859 conn(0x7f65200fe090 msgr2=0x7f65201004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 -- 172.21.15.102:0/2603016859 shutdown_connections 2024-04-25T14:56:20.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.915+0000 7f6527235700 1 -- 172.21.15.102:0/2603016859 wait complete. 2024-04-25T14:56:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.916+0000 7f6527235700 1 Processor -- start 2024-04-25T14:56:20.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.916+0000 7f6527235700 1 -- start start 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 0x7f6520198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6520104670 0x7f65201a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f652010e5e0 con 0x7f6520104670 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f652010e460 con 0x7f6520102ad0 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f652010e760 con 0x7f6520103cd0 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45136/0 (socket says 172.21.15.102:45136) 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 -- 172.21.15.102:0/962052686 learned_addr learned my addr 172.21.15.102:0/962052686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6524fd1700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 0x7f6520198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 -- 172.21.15.102:0/962052686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 msgr2=0x7f6520198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 0x7f6520198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 -- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6520104670 msgr2=0x7f65201a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6520104670 0x7f65201a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 -- 172.21.15.102:0/962052686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65201a0890 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651ffff700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6514006010 tx=0x7f65140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514027070 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65201a0b20 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6514004100 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.917+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514022d70 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.918+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65201a0fd0 con 0x7f6520103cd0 2024-04-25T14:56:20.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.919+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6520069730 con 0x7f6520103cd0 2024-04-25T14:56:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.920+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f651402a070 con 0x7f6520103cd0 2024-04-25T14:56:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.920+0000 7f651dffb700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 0x7f65080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.920+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f65140cfa70 con 0x7f6520103cd0 2024-04-25T14:56:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.921+0000 7f6524fd1700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 0x7f65080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:20.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.922+0000 7f6524fd1700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 0x7f65080a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f652019a120 tx=0x7f6518009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:20.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:20.925+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f651405eff0 con 0x7f6520103cd0 2024-04-25T14:56:21.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:21.131+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6520062e80 con 0x7f65080a3bb0 2024-04-25T14:56:21.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:20 smithi083 ceph-mon[32919]: osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] boot 2024-04-25T14:56:21.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:20 smithi083 ceph-mon[32919]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:56:21.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:20 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:21.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:20 smithi083 ceph-mon[32919]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:21.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:20 smithi102 ceph-mon[33005]: osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] boot 2024-04-25T14:56:21.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:20 smithi102 ceph-mon[33005]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:56:21.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:20 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:21.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:20 smithi102 ceph-mon[33005]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:21.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:20 smithi019 ceph-mon[28090]: osd.1 [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] boot 2024-04-25T14:56:21.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:20 smithi019 ceph-mon[28090]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:56:21.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:20 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:56:21.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:20 smithi019 ceph-mon[28090]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:56:22.031 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:21 smithi102 ceph-mon[33005]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:56:22.031 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:21 smithi102 ceph-mon[33005]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:22.031 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:22.031 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:22.031 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:22.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:21 smithi083 ceph-mon[32919]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:56:22.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:21 smithi083 ceph-mon[32919]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:22.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:22.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:22.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:21 smithi019 ceph-mon[28090]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:56:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:21 smithi019 ceph-mon[28090]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:23.045 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:22 smithi102 ceph-mon[33005]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:23.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:22 smithi083 ceph-mon[32919]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:22 smithi019 ceph-mon[28090]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:24.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:23 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/2044294860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:23 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:23 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]': finished 2024-04-25T14:56:24.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:23 smithi083 ceph-mon[32919]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:56:24.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:23 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:24.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:23 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/2044294860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:23 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:23 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]': finished 2024-04-25T14:56:24.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:23 smithi102 ceph-mon[33005]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:56:24.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:23 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:23 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/2044294860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:23 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]: dispatch 2024-04-25T14:56:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:23 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2326e2-31a4-4563-9b95-e5da5099e1f9"}]': finished 2024-04-25T14:56:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:23 smithi019 ceph-mon[28090]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:56:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:23 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:25.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:24 smithi083 ceph-mon[32919]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:25.293 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:24 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3695933810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:24 smithi102 ceph-mon[33005]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:24 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3695933810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:25.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:24 smithi019 ceph-mon[28090]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:25.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:24 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3695933810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:27.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:26 smithi019 ceph-mon[28090]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:27.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:26 smithi083 ceph-mon[32919]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:27.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:26 smithi102 ceph-mon[33005]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:29.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:28 smithi102 ceph-mon[33005]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:29.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:28 smithi083 ceph-mon[32919]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:29.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:28 smithi019 ceph-mon[28090]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:31.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:30 smithi083 ceph-mon[32919]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:31.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:30 smithi102 ceph-mon[33005]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:31.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:30 smithi019 ceph-mon[28090]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:56:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:31 smithi083 ceph-mon[32919]: Deploying daemon osd.2 on smithi102 2024-04-25T14:56:32.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:56:32.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:32.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:31 smithi102 ceph-mon[33005]: Deploying daemon osd.2 on smithi102 2024-04-25T14:56:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:56:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:31 smithi019 ceph-mon[28090]: Deploying daemon osd.2 on smithi102 2024-04-25T14:56:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:32 smithi102 ceph-mon[33005]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:33.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:32 smithi083 ceph-mon[32919]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:33.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:32 smithi019 ceph-mon[28090]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:35.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:34 smithi083 ceph-mon[32919]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:34 smithi102 ceph-mon[33005]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:35.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:34 smithi019 ceph-mon[28090]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:37.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:37 smithi083 ceph-mon[32919]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:37 smithi102 ceph-mon[33005]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:37 smithi019 ceph-mon[28090]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:39.488 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:39.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:39.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.915+0000 7f651dffb700 1 -- 172.21.15.102:0/962052686 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6520062e80 con 0x7f65080a3bb0 2024-04-25T14:56:39.917 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 msgr2=0x7f65080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 0x7f65080a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f652019a120 tx=0x7f6518009180 comp rx=0 tx=0).stop 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 msgr2=0x7f6520199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6514006010 tx=0x7f65140047b0 comp rx=0 tx=0).stop 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 shutdown_connections 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f65080a3bb0 0x7f65080a6070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6520104670 0x7f65201a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6520102ad0 0x7f6520198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:39.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 --2- 172.21.15.102:0/962052686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6520103cd0 0x7f6520199180 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:39.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 >> 172.21.15.102:0/962052686 conn(0x7f65200fe090 msgr2=0x7f6520107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:39.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 shutdown_connections 2024-04-25T14:56:39.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:56:39.917+0000 7f6527235700 1 -- 172.21.15.102:0/962052686 wait complete. 2024-04-25T14:56:40.520 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T14:56:40.522 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-04-25T14:56:40.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:56:40.975 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:56:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:41.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:41.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:40 smithi102 ceph-mon[33005]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:41.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:40 smithi019 ceph-mon[28090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:40 smithi083 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:56:42.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:41 smithi083 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:56:42.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:41 smithi083 ceph-mon[32919]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:42.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:41 smithi083 ceph-mon[32919]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:56:42.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:42.323 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:41 smithi083 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:42.323 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-mon[33005]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:56:42.324 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-mon[33005]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:42.324 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-mon[33005]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:56:42.324 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:42.324 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-mon[33005]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:42.324 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2[38151]: 2024-04-25T14:56:41.901+0000 7f74bff39700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T14:56:42.324 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2[38151]: 2024-04-25T14:56:41.906+0000 7f74bbd2c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:56:42.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:41 smithi019 ceph-mon[28090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:56:42.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:41 smithi019 ceph-mon[28090]: from='osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:42.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:41 smithi019 ceph-mon[28090]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:56:42.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:42.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:41 smithi019 ceph-mon[28090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:56:43.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:56:43.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:43.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:56:43.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:56:43.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:56:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:43.971 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:56:43.972 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:56:43.972 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T14:56:43.972 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T14:56:43.972 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0364179 s, 288 MB/s 2024-04-25T14:56:43.972 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] boot 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi102 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:44.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-25T14:56:44.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:44.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:44.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] boot 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi102 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-25T14:56:44.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:44.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:44.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: osd.2 [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] boot 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi102 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-25T14:56:44.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:44.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:44.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:44.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-04-25T14:56:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:44 smithi019 ceph-mon[28090]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:44 smithi019 ceph-mon[28090]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:56:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:45.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:44 smithi083 ceph-mon[32919]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:45.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:44 smithi083 ceph-mon[32919]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:56:45.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:44 smithi102 ceph-mon[33005]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:44 smithi102 ceph-mon[33005]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:56:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:56:46.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:45 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:46.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:45 smithi019 ceph-mon[28090]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:56:46.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:45 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:46.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:45 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:46.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:45 smithi083 ceph-mon[32919]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:56:46.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:45 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:46.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:45 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:46.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:45 smithi102 ceph-mon[33005]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:56:46.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:45 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47377]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.520 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47383]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.654 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.654 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.654 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41624]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.654 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41624]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.654 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41627]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.655 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.655 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41627]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.655 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41627]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42221]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42221]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42221]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42221]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42224]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42224]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42215]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42218]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42218]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 14:56:46 smithi102 sudo[42218]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47386]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47386]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47389]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 sudo[47389]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.951 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41630]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41630]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41630]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41633]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:56:46.952 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 sudo[41633]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:46 smithi019 ceph-mon[28090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:46 smithi083 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:56:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:56:47.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:56:47.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:46 smithi102 ceph-mon[33005]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 -- 172.21.15.19:0/2252066815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 msgr2=0x7f90f810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 --2- 172.21.15.19:0/2252066815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f810d130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b0d0 tx=0x7f90f000b3e0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 -- 172.21.15.19:0/2252066815 shutdown_connections 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 --2- 172.21.15.19:0/2252066815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f810d130 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 --2- 172.21.15.19:0/2252066815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f8105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 --2- 172.21.15.19:0/2252066815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 0x7f90f8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 -- 172.21.15.19:0/2252066815 >> 172.21.15.19:0/2252066815 conn(0x7f90f81002a0 msgr2=0x7f90f81026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 -- 172.21.15.19:0/2252066815 shutdown_connections 2024-04-25T14:56:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.564+0000 7f9100400700 1 -- 172.21.15.19:0/2252066815 wait complete. 2024-04-25T14:56:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.565+0000 7f9100400700 1 Processor -- start 2024-04-25T14:56:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.565+0000 7f9100400700 1 -- start start 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 0x7f90f819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f81a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81107b0 con 0x7f90f8106880 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8110630 con 0x7f90f8105ee0 2024-04-25T14:56:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f9100400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8110930 con 0x7f90f8104ce0 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fe400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f81a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fda00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fe400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f81a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53226/0 (socket says 172.21.15.19:53226) 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fda00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:36836/0 (socket says 172.21.15.19:36836) 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fe400700 1 -- 172.21.15.19:0/40238390 learned_addr learned my addr 172.21.15.19:0/40238390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:56:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.566+0000 7f90fd000700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 0x7f90f819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 -- 172.21.15.19:0/40238390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 msgr2=0x7f90f819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 0x7f90f819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 -- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 msgr2=0x7f90f81a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f81a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 -- 172.21.15.19:0/40238390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f81a2a00 con 0x7f90f8104ce0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90fda00700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00e9c0 tx=0x7f90ec00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec004220 con 0x7f90f8104ce0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f81a2cf0 con 0x7f90f8104ce0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ec0043c0 con 0x7f90f8104ce0 2024-04-25T14:56:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec003c50 con 0x7f90f8104ce0 2024-04-25T14:56:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.567+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f81a31c0 con 0x7f90f8104ce0 2024-04-25T14:56:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.569+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f8069730 con 0x7f90f8104ce0 2024-04-25T14:56:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.571+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f90ec00c7c0 con 0x7f90f8104ce0 2024-04-25T14:56:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.572+0000 7f90f6000700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 0x7f90e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.572+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f90ec0cf180 con 0x7f90f8104ce0 2024-04-25T14:56:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.572+0000 7f90fd000700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 0x7f90e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.573+0000 7f90fd000700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 0x7f90e00a6070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009c60 tx=0x7f90e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.577+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90ec09cf70 con 0x7f90f8104ce0 2024-04-25T14:56:47.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:47.818+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f90f8062e80 con 0x7f90e00a3bb0 2024-04-25T14:56:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:47 smithi019 ceph-mon[28090]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:56:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:48.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:47 smithi083 ceph-mon[32919]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:56:48.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:48.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:48.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:47 smithi102 ceph-mon[33005]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:56:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:56:48.984+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f90ec098ee0 con 0x7f90f8104ce0 2024-04-25T14:56:49.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:48 smithi019 ceph-mon[28090]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:49.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:48 smithi019 ceph-mon[28090]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:48 smithi019 ceph-mon[28090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:56:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:48 smithi019 ceph-mon[28090]: Cluster is now healthy 2024-04-25T14:56:49.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:48 smithi083 ceph-mon[32919]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:49.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:48 smithi083 ceph-mon[32919]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:49.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:48 smithi083 ceph-mon[32919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:56:49.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:48 smithi083 ceph-mon[32919]: Cluster is now healthy 2024-04-25T14:56:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:48 smithi102 ceph-mon[33005]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:48 smithi102 ceph-mon[33005]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:48 smithi102 ceph-mon[33005]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:56:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:48 smithi102 ceph-mon[33005]: Cluster is now healthy 2024-04-25T14:56:50.217 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:49 smithi019 ceph-mon[28090]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:56:50.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:49 smithi083 ceph-mon[32919]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:56:50.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:49 smithi102 ceph-mon[33005]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:56:51.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:50 smithi083 ceph-mon[32919]: pgmap v99: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:51.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:50 smithi102 ceph-mon[33005]: pgmap v99: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:51.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:50 smithi019 ceph-mon[28090]: pgmap v99: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:51 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]: dispatch 2024-04-25T14:56:52.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:51 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]': finished 2024-04-25T14:56:52.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:51 smithi083 ceph-mon[32919]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:56:52.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:56:52.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:51 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]: dispatch 2024-04-25T14:56:52.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:51 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]': finished 2024-04-25T14:56:52.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:51 smithi102 ceph-mon[33005]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:56:52.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:56:52.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:51 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]: dispatch 2024-04-25T14:56:52.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:51 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2857104212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05431369-edfd-411e-9a63-b6b60b86ae1d"}]': finished 2024-04-25T14:56:52.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:51 smithi019 ceph-mon[28090]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:56:52.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:56:53.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:53 smithi102 ceph-mon[33005]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:53.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:53 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2343876357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:53 smithi019 ceph-mon[28090]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:53 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2343876357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:53.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:53 smithi083 ceph-mon[32919]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:53.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:53 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2343876357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:55.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:55 smithi102 ceph-mon[33005]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:55.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:55 smithi019 ceph-mon[28090]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:55.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:55 smithi083 ceph-mon[32919]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:57 smithi102 ceph-mon[33005]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:57 smithi019 ceph-mon[28090]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:57.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:57 smithi083 ceph-mon[32919]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:59.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:56:59 smithi083 ceph-mon[32919]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:59.314 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:56:59 smithi019 ceph-mon[28090]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:56:59.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:56:59 smithi102 ceph-mon[33005]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:01.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:01 smithi019 ceph-mon[28090]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:01.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:01 smithi083 ceph-mon[32919]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:01.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:01 smithi102 ceph-mon[33005]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:02.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:57:02.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:02 smithi019 ceph-mon[28090]: Deploying daemon osd.3 on smithi019 2024-04-25T14:57:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:57:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:02 smithi083 ceph-mon[32919]: Deploying daemon osd.3 on smithi019 2024-04-25T14:57:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:57:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:02 smithi102 ceph-mon[33005]: Deploying daemon osd.3 on smithi019 2024-04-25T14:57:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:03 smithi019 ceph-mon[28090]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:03.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:03 smithi083 ceph-mon[32919]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:03 smithi102 ceph-mon[33005]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:05.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:05 smithi019 ceph-mon[28090]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:05.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:05 smithi083 ceph-mon[32919]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:05 smithi102 ceph-mon[33005]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:07.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:07 smithi019 ceph-mon[28090]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:07.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:07 smithi083 ceph-mon[32919]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:07 smithi102 ceph-mon[33005]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:09.359 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:09 smithi102 ceph-mon[33005]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:09.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:09 smithi019 ceph-mon[28090]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:09 smithi083 ceph-mon[32919]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:11.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:11 smithi019 ceph-mon[28090]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:11.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:11.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:11 smithi083 ceph-mon[32919]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:11.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:11 smithi102 ceph-mon[33005]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:13.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:13 smithi019 ceph-mon[28090]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:13.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:13 smithi083 ceph-mon[32919]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:13 smithi102 ceph-mon[33005]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:15.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:15 smithi019 ceph-mon[28090]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:15.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:15 smithi019 ceph-mon[28090]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:15 smithi019 ceph-mon[28090]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:15 smithi083 ceph-mon[32919]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:15 smithi083 ceph-mon[32919]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:15 smithi083 ceph-mon[32919]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:15 smithi102 ceph-mon[33005]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:15 smithi102 ceph-mon[33005]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:15 smithi102 ceph-mon[33005]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:57:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.718+0000 7f90f6000700 1 -- 172.21.15.19:0/40238390 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f90f8062e80 con 0x7f90e00a3bb0 2024-04-25T14:57:15.721 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.721+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 msgr2=0x7f90e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.721+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 0x7f90e00a6070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009c60 tx=0x7f90e4009260 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.721+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 msgr2=0x7f90f819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.721+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00e9c0 tx=0x7f90ec00ecd0 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 shutdown_connections 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90e00a3bb0 0x7f90e00a6070 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90f8106880 0x7f90f81a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8104ce0 0x7f90f819adb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 --2- 172.21.15.19:0/40238390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90f8105ee0 0x7f90f819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 >> 172.21.15.19:0/40238390 conn(0x7f90f81002a0 msgr2=0x7f90f8109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 shutdown_connections 2024-04-25T14:57:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:57:15.722+0000 7f9100400700 1 -- 172.21.15.19:0/40238390 wait complete. 2024-04-25T14:57:16.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:57:16.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:16.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.528 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T14:57:16.532 INFO:tasks.cephadm:Deploying osd.4 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-25T14:57:16.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:57:16.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:57:16.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:16.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:16.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:16.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:16.774 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:57:17.228 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] boot 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:57:17.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] boot 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:57:17.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: osd.3 [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] boot 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:57:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:57:18.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:18 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:57:18.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:18 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:57:18.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:18 smithi019 ceph-mon[28090]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:57:18.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:18 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:57:18.260 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:18 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:57:18.260 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:18 smithi083 ceph-mon[32919]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:57:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:18 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:57:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:18 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:57:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:18 smithi102 ceph-mon[33005]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:57:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:57:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:57:18.641 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T14:57:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T14:57:18.642 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112619 s, 931 MB/s 2024-04-25T14:57:18.642 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-04-25T14:57:18.642 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T14:57:19.232 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:19 smithi083 ceph-mon[32919]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:19.232 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:19 smithi083 ceph-mon[32919]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:57:19.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-25T14:57:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:19 smithi019 ceph-mon[28090]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:19 smithi019 ceph-mon[28090]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:57:19.516 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:57:19.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:19 smithi102 ceph-mon[33005]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:19.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:19 smithi102 ceph-mon[33005]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 -- 172.21.15.83:0/3487837072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88102ba0 msgr2=0x7f9e88103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/3487837072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88102ba0 0x7f9e88103000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0099c0 tx=0x7f9e7c009cd0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 -- 172.21.15.83:0/3487837072 shutdown_connections 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/3487837072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e88103540 0x7f9e88109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/3487837072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e881019a0 0x7f9e88101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/3487837072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88102ba0 0x7f9e88103000 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 -- 172.21.15.83:0/3487837072 >> 172.21.15.83:0/3487837072 conn(0x7f9e880fcf40 msgr2=0x7f9e880ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 -- 172.21.15.83:0/3487837072 shutdown_connections 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.015+0000 7f9e8fb38700 1 -- 172.21.15.83:0/3487837072 wait complete. 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 Processor -- start 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 -- start start 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e881019a0 0x7f9e88197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e88102ba0 0x7f9e88198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d530 con 0x7f9e88102ba0 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d3b0 con 0x7f9e88103540 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8fb38700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810d6b0 con 0x7f9e881019a0 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51352/0 (socket says 172.21.15.83:51352) 2024-04-25T14:57:21.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 -- 172.21.15.83:0/613160995 learned_addr learned my addr 172.21.15.83:0/613160995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8d8d4700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e881019a0 0x7f9e88197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 -- 172.21.15.83:0/613160995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e881019a0 msgr2=0x7f9e88197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e881019a0 0x7f9e88197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 -- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e88102ba0 msgr2=0x7f9e88198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e88102ba0 0x7f9e88198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.016+0000 7f9e8e0d5700 1 -- 172.21.15.83:0/613160995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819f870 con 0x7f9e88103540 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e8e0d5700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800d960 tx=0x7f9e7800dc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7801b070 con 0x7f9e88103540 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7800ebd0 con 0x7f9e88103540 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780145a0 con 0x7f9e88103540 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8806a780 con 0x7f9e88103540 2024-04-25T14:57:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.017+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8806acf0 con 0x7f9e88103540 2024-04-25T14:57:21.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.018+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88069730 con 0x7f9e88103540 2024-04-25T14:57:21.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.019+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e7801e070 con 0x7f9e88103540 2024-04-25T14:57:21.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.019+0000 7f9e86ffd700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:21.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.020+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f9e780d08d0 con 0x7f9e88103540 2024-04-25T14:57:21.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.020+0000 7f9e8d8d4700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:21.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.021+0000 7f9e8d8d4700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 0x7f9e700a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e88199100 tx=0x7f9e8000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:21.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.024+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e7809e4d0 con 0x7f9e88103540 2024-04-25T14:57:21.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:21.235+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9e8806b010 con 0x7f9e700a3c40 2024-04-25T14:57:21.401 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:21 smithi019 ceph-mon[28090]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:21.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:21 smithi019 ceph-mon[28090]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:57:21.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:21 smithi083 ceph-mon[32919]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:21.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:21 smithi083 ceph-mon[32919]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:57:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:21 smithi102 ceph-mon[33005]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:21 smithi102 ceph-mon[33005]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:57:22.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:57:22.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:22.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:22.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:22.512 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:22.513 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:22.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:22.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:23.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:23 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi019 2024-04-25T14:57:23.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:23 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 8547M 2024-04-25T14:57:23.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:23 smithi083 ceph-mon[32919]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:57:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:23 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi019 2024-04-25T14:57:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:23 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 8547M 2024-04-25T14:57:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:23 smithi102 ceph-mon[33005]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:57:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:23 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi019 2024-04-25T14:57:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:23 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 8547M 2024-04-25T14:57:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:23 smithi019 ceph-mon[28090]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:57:24.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:24 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/2644565068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:24 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:24 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]': finished 2024-04-25T14:57:24.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:24 smithi083 ceph-mon[32919]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:57:24.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:24 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:24 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/2644565068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:24 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:24 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]': finished 2024-04-25T14:57:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:24 smithi102 ceph-mon[33005]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:57:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:24 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:24 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/2644565068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:24 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]: dispatch 2024-04-25T14:57:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:24 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45e8783f-267c-42c5-834c-905d5e715a05"}]': finished 2024-04-25T14:57:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:24 smithi019 ceph-mon[28090]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:57:24.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:24 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:25.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:25 smithi019 ceph-mon[28090]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T14:57:25.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:25 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/2353203680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:25.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:25 smithi083 ceph-mon[32919]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T14:57:25.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:25 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/2353203680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:25.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:25 smithi102 ceph-mon[33005]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T14:57:25.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:25 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/2353203680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:27 smithi083 ceph-mon[32919]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:27 smithi102 ceph-mon[33005]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:27 smithi019 ceph-mon[28090]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:29.419 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:29 smithi083 ceph-mon[32919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:57:29.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:29 smithi102 ceph-mon[33005]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:57:29.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:29 smithi019 ceph-mon[28090]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:57:31.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:31 smithi083 ceph-mon[32919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:31 smithi102 ceph-mon[33005]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:31.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:31 smithi019 ceph-mon[28090]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:32.479 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:32 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:57:32.480 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:32 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:32.480 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:32 smithi083 ceph-mon[32919]: Deploying daemon osd.4 on smithi083 2024-04-25T14:57:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:32 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:57:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:32 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:32 smithi102 ceph-mon[33005]: Deploying daemon osd.4 on smithi083 2024-04-25T14:57:32.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:57:32.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:32.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:32 smithi019 ceph-mon[28090]: Deploying daemon osd.4 on smithi083 2024-04-25T14:57:33.409 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:33 smithi083 ceph-mon[32919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:33.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:33 smithi102 ceph-mon[33005]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:33.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:33 smithi019 ceph-mon[28090]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:35.435 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:35 smithi083 ceph-mon[32919]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:35.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:35 smithi102 ceph-mon[33005]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:35.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:35 smithi019 ceph-mon[28090]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:37.527 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:37 smithi083 ceph-mon[32919]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:37.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.528 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:37 smithi102 ceph-mon[33005]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:37 smithi019 ceph-mon[28090]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:39.432 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:39 smithi083 ceph-mon[32919]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:39 smithi102 ceph-mon[33005]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:39 smithi019 ceph-mon[28090]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:40.471 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 4 on host 'smithi083' 2024-04-25T14:57:40.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.470+0000 7f9e86ffd700 1 -- 172.21.15.83:0/613160995 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e8806b010 con 0x7f9e700a3c40 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 msgr2=0x7f9e700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 0x7f9e700a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e88199100 tx=0x7f9e8000d040 comp rx=0 tx=0).stop 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 msgr2=0x7f9e8819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800d960 tx=0x7f9e7800dc70 comp rx=0 tx=0).stop 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 shutdown_connections 2024-04-25T14:57:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e88103540 0x7f9e8819f1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e881019a0 0x7f9e88197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 --2- 172.21.15.83:0/613160995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e88102ba0 0x7f9e88198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 >> 172.21.15.83:0/613160995 conn(0x7f9e880fcf40 msgr2=0x7f9e88105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.472+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 shutdown_connections 2024-04-25T14:57:40.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:57:40.473+0000 7f9e8fb38700 1 -- 172.21.15.83:0/613160995 wait complete. 2024-04-25T14:57:40.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:40.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:41.033 DEBUG:teuthology.orchestra.run.smithi083:osd.4> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T14:57:41.035 INFO:tasks.cephadm:Deploying osd.5 on smithi102 with /dev/vg_nvme/lv_3... 2024-04-25T14:57:41.035 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:57:41.271 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:57:41.677 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:41 smithi102 ceph-mon[33005]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:41.677 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:41 smithi102 ceph-mon[33005]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:41.677 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:41 smithi102 ceph-mon[33005]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:41 smithi019 ceph-mon[28090]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:41 smithi019 ceph-mon[28090]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:41 smithi019 ceph-mon[28090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:41.878 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:41 smithi083 ceph-mon[32919]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:41.879 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:41 smithi083 ceph-mon[32919]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:41.879 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:41 smithi083 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:42.642 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:42 smithi102 ceph-mon[33005]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:57:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:57:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:57:42.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:42.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:42.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:42 smithi019 ceph-mon[28090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: from='osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:57:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:42 smithi083 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130541 s, 803 MB/s 2024-04-25T14:57:43.181 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-25T14:57:43.621 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T14:57:43.477+0000 7f92822f3700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T14:57:43.621 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T14:57:43.586+0000 7f927a0de700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:57:43.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:43 smithi102 ceph-mon[33005]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:43.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:43 smithi102 ceph-mon[33005]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:57:43.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:43 smithi102 ceph-mon[33005]: from='osd.4 ' entity='osd.4' 2024-04-25T14:57:43.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:43 smithi019 ceph-mon[28090]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:43.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:43 smithi019 ceph-mon[28090]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:57:43.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:43 smithi019 ceph-mon[28090]: from='osd.4 ' entity='osd.4' 2024-04-25T14:57:43.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.882 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-04-25T14:57:43.986 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-mon[32919]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:43.986 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-mon[32919]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:57:43.986 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:43.986 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-mon[32919]: from='osd.4 ' entity='osd.4' 2024-04-25T14:57:43.986 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:44.118 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:57:45.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] boot 2024-04-25T14:57:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:57:45.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:45.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:57:45.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:57:45.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] boot 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:57:45.052 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:57:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: osd.4 [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] boot 2024-04-25T14:57:45.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:57:45.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 -- 172.21.15.102:0/3876245805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 msgr2=0x7f626410b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 --2- 172.21.15.102:0/3876245805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f626410b0c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b0a0 tx=0x7f625c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 -- 172.21.15.102:0/3876245805 shutdown_connections 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 --2- 172.21.15.102:0/3876245805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f626410b0c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 --2- 172.21.15.102:0/3876245805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 0x7f62641044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 --2- 172.21.15.102:0/3876245805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6264100b50 0x7f6264100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 -- 172.21.15.102:0/3876245805 >> 172.21.15.102:0/3876245805 conn(0x7f62640fc710 msgr2=0x7f62640feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 -- 172.21.15.102:0/3876245805 shutdown_connections 2024-04-25T14:57:45.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.574+0000 7f626b19e700 1 -- 172.21.15.102:0/3876245805 wait complete. 2024-04-25T14:57:45.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 Processor -- start 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 -- start start 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6264100b50 0x7f6264198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 0x7f62641a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f626410e540 con 0x7f6264100b50 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f626410e3c0 con 0x7f62641040d0 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f626410e6c0 con 0x7f62641014f0 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37708/0 (socket says 172.21.15.102:37708) 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 -- 172.21.15.102:0/2945809943 learned_addr learned my addr 172.21.15.102:0/2945809943 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626973b700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 0x7f62641a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:45.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 -- 172.21.15.102:0/2945809943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 msgr2=0x7f62641a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 0x7f62641a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 -- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6264100b50 msgr2=0x7f6264198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6264100b50 0x7f6264198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 -- 172.21.15.102:0/2945809943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62641a0780 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6263fff700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f625000b560 tx=0x7f625000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250015020 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.575+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62641a0a70 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.576+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250011ce0 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.576+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250010d20 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.576+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a1000 con 0x7f62641014f0 2024-04-25T14:57:45.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.577+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6264069770 con 0x7f62641014f0 2024-04-25T14:57:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.578+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6250018020 con 0x7f62641014f0 2024-04-25T14:57:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.578+0000 7f6261ffb700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 0x7f62540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.578+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f62500cea50 con 0x7f62641014f0 2024-04-25T14:57:45.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.579+0000 7f6268f3a700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 0x7f62540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:45.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.580+0000 7f6268f3a700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 0x7f62540a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f626406aeb0 tx=0x7f6258005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:45.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.583+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f625001f020 con 0x7f62641014f0 2024-04-25T14:57:45.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:57:45.792+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6264062e80 con 0x7f62540a3cc0 2024-04-25T14:57:46.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:45 smithi083 ceph-mon[32919]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:46.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:45 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi083 2024-04-25T14:57:46.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:45 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T14:57:46.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:45 smithi102 ceph-mon[33005]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:46.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:45 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi083 2024-04-25T14:57:46.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:45 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T14:57:46.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:45 smithi019 ceph-mon[28090]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:57:46.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:45 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi083 2024-04-25T14:57:46.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:45 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T14:57:46.724 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:46 smithi102 ceph-mon[33005]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:57:46.725 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:46.725 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:46.725 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:46 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:47.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:46 smithi083 ceph-mon[32919]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:57:47.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:47.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:47.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:46 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:47.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:46 smithi019 ceph-mon[28090]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:57:47.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:47.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:47.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:46 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:48.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:47 smithi083 ceph-mon[32919]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:48.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:47 smithi083 ceph-mon[32919]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:47 smithi102 ceph-mon[33005]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:47 smithi102 ceph-mon[33005]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:47 smithi019 ceph-mon[28090]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:47 smithi019 ceph-mon[28090]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:48.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:48 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3500501490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:48.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:48 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:48.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:48 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]': finished 2024-04-25T14:57:48.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:48 smithi083 ceph-mon[32919]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:57:48.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:48 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:57:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:48 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3500501490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:48 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:48 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]': finished 2024-04-25T14:57:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:48 smithi102 ceph-mon[33005]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:57:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:48 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:57:49.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:48 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3500501490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:49.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:48 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]: dispatch 2024-04-25T14:57:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:48 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23ba29b0-bb4a-458a-b988-de82bd6f7629"}]': finished 2024-04-25T14:57:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:48 smithi019 ceph-mon[28090]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:57:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:48 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:57:49.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:49 smithi083 ceph-mon[32919]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:49.958 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:49 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3090157695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:50.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:49 smithi102 ceph-mon[33005]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:50.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:49 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3090157695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:49 smithi019 ceph-mon[28090]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:49 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3090157695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:52.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:51 smithi083 ceph-mon[32919]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:51 smithi102 ceph-mon[33005]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:52.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:51 smithi019 ceph-mon[28090]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:53.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:53 smithi102 ceph-mon[33005]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:54.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:53 smithi083 ceph-mon[32919]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:54.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:53 smithi019 ceph-mon[28090]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:56.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:55 smithi083 ceph-mon[32919]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:55 smithi102 ceph-mon[33005]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:55 smithi019 ceph-mon[28090]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:56.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:57:56.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:57.023 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:57:57.023 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:57.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:57:57.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:57.928 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:57 smithi102 ceph-mon[33005]: Deploying daemon osd.5 on smithi102 2024-04-25T14:57:57.928 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:57 smithi102 ceph-mon[33005]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:58.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:57 smithi083 ceph-mon[32919]: Deploying daemon osd.5 on smithi102 2024-04-25T14:57:58.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:57 smithi083 ceph-mon[32919]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:58.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:57 smithi019 ceph-mon[28090]: Deploying daemon osd.5 on smithi102 2024-04-25T14:57:58.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:57 smithi019 ceph-mon[28090]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:57:59.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:57:59 smithi102 ceph-mon[33005]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:00.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:57:59 smithi083 ceph-mon[32919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:00.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:57:59 smithi019 ceph-mon[28090]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:02.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:01 smithi102 ceph-mon[33005]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:02.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:01 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:01 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:01 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:02.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:01 smithi083 ceph-mon[32919]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:02.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:01 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:01 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:01 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:02.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:01 smithi019 ceph-mon[28090]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:02.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:02.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:04.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:03 smithi083 ceph-mon[32919]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:04.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:03 smithi102 ceph-mon[33005]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:04.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:03 smithi019 ceph-mon[28090]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:04.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.731+0000 7f6261ffb700 1 -- 172.21.15.102:0/2945809943 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6264062e80 con 0x7f62540a3cc0 2024-04-25T14:58:04.733 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 5 on host 'smithi102' 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 msgr2=0x7f62540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 0x7f62540a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f626406aeb0 tx=0x7f6258005ca0 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 msgr2=0x7f6264199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f625000b560 tx=0x7f625000b870 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 shutdown_connections 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62641040d0 0x7f62641a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f62540a3cc0 0x7f62540a6180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6264100b50 0x7f6264198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 --2- 172.21.15.102:0/2945809943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62641014f0 0x7f6264199060 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.733+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 >> 172.21.15.102:0/2945809943 conn(0x7f62640fc710 msgr2=0x7f6264107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:04.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.734+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 shutdown_connections 2024-04-25T14:58:04.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:58:04.734+0000 7f626b19e700 1 -- 172.21.15.102:0/2945809943 wait complete. 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:04.947 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:04 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:05.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:04 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:05.309 DEBUG:teuthology.orchestra.run.smithi102:osd.5> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T14:58:05.311 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-04-25T14:58:05.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:58:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:05.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:05.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:05.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:04 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:05.773 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:58:06.207 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:05 smithi019 ceph-mon[28090]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:06.207 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:05 smithi019 ceph-mon[28090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:06.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:05 smithi083 ceph-mon[32919]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:06.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:05 smithi083 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:06.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:05 smithi102 ceph-mon[33005]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:06.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:05 smithi102 ceph-mon[33005]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:58:07.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:58:07.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:58:07.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:07.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:06 smithi083 ceph-mon[32919]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:07.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:58:07.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:58:07.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:07.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-mon[33005]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:07.308 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T14:58:06.921+0000 7f2b495f7700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T14:58:07.308 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 14:58:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T14:58:06.928+0000 7f2b453ea700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:58:07.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:58:07.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: from='osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:58:07.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:07.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:58:07.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:06 smithi019 ceph-mon[28090]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] boot 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:58:08.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:58:08.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:58:08.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] boot 2024-04-25T14:58:08.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:58:08.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:58:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:58:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: osd.5 [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] boot 2024-04-25T14:58:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:58:08.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:58:08.951 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:58:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:58:08.952 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T14:58:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T14:58:08.952 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0359176 s, 292 MB/s 2024-04-25T14:58:08.952 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T14:58:09.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi102 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-25T14:58:09.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:09.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:09.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi102 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:09.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi102 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:09.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:09.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-04-25T14:58:10.316 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:58:10.472 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:10 smithi019 ceph-mon[28090]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:58:10.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:10 smithi102 ceph-mon[33005]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:58:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:10 smithi083 ceph-mon[32919]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:58:11.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:11 smithi019 ceph-mon[28090]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:11.452 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:11 smithi019 ceph-mon[28090]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:58:11.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:11 smithi083 ceph-mon[32919]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:11.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:11 smithi083 ceph-mon[32919]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:58:11.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:11 smithi102 ceph-mon[33005]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:11.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:11 smithi102 ceph-mon[33005]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:58:12.439 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:12 smithi019 ceph-mon[28090]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:58:12.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:12 smithi083 ceph-mon[32919]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:58:12.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:12 smithi102 ceph-mon[33005]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:58:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 -- 172.21.15.19:0/2155173783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 msgr2=0x7f38f4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 --2- 172.21.15.19:0/2155173783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f4102b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0098f0 tx=0x7f38ec009c00 comp rx=0 tx=0).stop 2024-04-25T14:58:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 -- 172.21.15.19:0/2155173783 shutdown_connections 2024-04-25T14:58:12.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 --2- 172.21.15.19:0/2155173783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f4102b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 --2- 172.21.15.19:0/2155173783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 --2- 172.21.15.19:0/2155173783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 -- 172.21.15.19:0/2155173783 >> 172.21.15.19:0/2155173783 conn(0x7f38f40fcf40 msgr2=0x7f38f40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 -- 172.21.15.19:0/2155173783 shutdown_connections 2024-04-25T14:58:12.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.764+0000 7f38fc200700 1 -- 172.21.15.19:0/2155173783 wait complete. 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.765+0000 7f38fc200700 1 Processor -- start 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.765+0000 7f38fc200700 1 -- start start 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4110750 con 0x7f38f41026a0 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41105d0 con 0x7f38f4101ce0 2024-04-25T14:58:12.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fc200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41108d0 con 0x7f38f4101380 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fa200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38f9800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38f9800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:33266/0 (socket says 172.21.15.19:33266) 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38f9800700 1 -- 172.21.15.19:0/3008122876 learned_addr learned my addr 172.21.15.19:0/3008122876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38fa200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39048/0 (socket says 172.21.15.19:39048) 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.766+0000 7f38f8e00700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 -- 172.21.15.19:0/3008122876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 msgr2=0x7f38f419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 -- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 msgr2=0x7f38f41a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 -- 172.21.15.19:0/3008122876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f41a2a90 con 0x7f38f4101ce0 2024-04-25T14:58:12.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38fa200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f8e00700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f419b370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38e000e9c0 tx=0x7f38e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e000c9c0 con 0x7f38f4101ce0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e000cb60 con 0x7f38f4101ce0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.767+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f41a2d80 con 0x7f38f4101ce0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.768+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0018420 con 0x7f38f4101ce0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.768+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f41a3240 con 0x7f38f4101ce0 2024-04-25T14:58:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.769+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f4069730 con 0x7f38f4101ce0 2024-04-25T14:58:12.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.771+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38e0010430 con 0x7f38f4101ce0 2024-04-25T14:58:12.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.772+0000 7f38f2000700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 0x7f38dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:12.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.772+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f38e00cf580 con 0x7f38f4101ce0 2024-04-25T14:58:12.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.772+0000 7f38f9800700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 0x7f38dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:12.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.773+0000 7f38f9800700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 0x7f38dc0a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009550 tx=0x7f38e8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:12.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:12.777+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38e009cd80 con 0x7f38f4101ce0 2024-04-25T14:58:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:13.019+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f38f4062e80 con 0x7f38dc0a3cc0 2024-04-25T14:58:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:13 smithi019 ceph-mon[28090]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:58:13.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:13.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:13.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:13.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:13 smithi083 ceph-mon[32919]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:58:13.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:13.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:13.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:13 smithi102 ceph-mon[33005]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:58:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:14.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:14 smithi019 ceph-mon[28090]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:14.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:14 smithi083 ceph-mon[32919]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:14 smithi102 ceph-mon[33005]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:15.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:15 smithi083 ceph-mon[32919]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T14:58:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:15 smithi102 ceph-mon[33005]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T14:58:15.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:15 smithi019 ceph-mon[28090]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T14:58:17.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:17 smithi083 ceph-mon[32919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T14:58:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:17 smithi102 ceph-mon[33005]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T14:58:17.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:17 smithi019 ceph-mon[28090]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T14:58:18.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]: dispatch 2024-04-25T14:58:18.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]': finished 2024-04-25T14:58:18.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:18 smithi083 ceph-mon[32919]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:58:18.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:18 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:18.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3046584984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]: dispatch 2024-04-25T14:58:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]': finished 2024-04-25T14:58:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:18 smithi102 ceph-mon[33005]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:58:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:18 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3046584984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:18.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]: dispatch 2024-04-25T14:58:18.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1025512663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e6a2a9-70b8-48a9-8dff-2b6f2d164734"}]': finished 2024-04-25T14:58:18.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:18 smithi019 ceph-mon[28090]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:58:18.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:18 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:18.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3046584984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:19.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:19 smithi083 ceph-mon[32919]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:58:19.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:19 smithi102 ceph-mon[33005]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:58:19.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:19 smithi019 ceph-mon[28090]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T14:58:21.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:21 smithi083 ceph-mon[32919]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:21 smithi102 ceph-mon[33005]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:21.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:21 smithi019 ceph-mon[28090]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:23.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:23 smithi083 ceph-mon[32919]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:23 smithi102 ceph-mon[33005]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:23 smithi019 ceph-mon[28090]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:25.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:25 smithi019 ceph-mon[28090]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:25.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:25 smithi083 ceph-mon[32919]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:25.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:25 smithi102 ceph-mon[33005]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:27.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:27 smithi019 ceph-mon[28090]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:27.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:58:27.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:27.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:27 smithi083 ceph-mon[32919]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:58:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:27 smithi102 ceph-mon[33005]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:58:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:28.434 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:28 smithi019 ceph-mon[28090]: Deploying daemon osd.6 on smithi019 2024-04-25T14:58:28.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:28 smithi083 ceph-mon[32919]: Deploying daemon osd.6 on smithi019 2024-04-25T14:58:28.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:28 smithi102 ceph-mon[33005]: Deploying daemon osd.6 on smithi019 2024-04-25T14:58:29.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:29 smithi019 ceph-mon[28090]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:29.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:29 smithi083 ceph-mon[32919]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:29.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:29 smithi102 ceph-mon[33005]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:31.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:31 smithi083 ceph-mon[32919]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:31 smithi102 ceph-mon[33005]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:31.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:31 smithi019 ceph-mon[28090]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:33.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:33 smithi083 ceph-mon[32919]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:33.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:33 smithi102 ceph-mon[33005]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:33.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:33 smithi019 ceph-mon[28090]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:35.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:35 smithi019 ceph-mon[28090]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:35.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:35 smithi102 ceph-mon[33005]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:35.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:35 smithi083 ceph-mon[32919]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:37.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:36 smithi019 ceph-mon[28090]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:37.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:36 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:36 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:36 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:37.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:36 smithi083 ceph-mon[32919]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:37.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:36 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:36 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:36 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:37.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:36 smithi102 ceph-mon[33005]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:37.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:36 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:36 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:37.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:36 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:39.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:38 smithi019 ceph-mon[28090]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:39.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:38 smithi083 ceph-mon[32919]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:39.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:38 smithi102 ceph-mon[33005]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:40 smithi102 ceph-mon[33005]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:40 smithi102 ceph-mon[33005]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:40 smithi102 ceph-mon[33005]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:41.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:40 smithi019 ceph-mon[28090]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:40 smithi019 ceph-mon[28090]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:40 smithi019 ceph-mon[28090]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:41.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.234+0000 7f38f2000700 1 -- 172.21.15.19:0/3008122876 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38f4062e80 con 0x7f38dc0a3cc0 2024-04-25T14:58:41.236 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 msgr2=0x7f38dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 0x7f38dc0a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009550 tx=0x7f38e8009040 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 msgr2=0x7f38f419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f419b370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38e000e9c0 tx=0x7f38e000ecd0 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 shutdown_connections 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38dc0a3cc0 0x7f38dc0a6180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38f41026a0 0x7f38f41a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38f4101380 0x7f38f419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 --2- 172.21.15.19:0/3008122876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38f4101ce0 0x7f38f419b370 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.238+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 >> 172.21.15.19:0/3008122876 conn(0x7f38f40fcf40 msgr2=0x7f38f41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.239+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 shutdown_connections 2024-04-25T14:58:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:58:41.239+0000 7f38fc200700 1 -- 172.21.15.19:0/3008122876 wait complete. 2024-04-25T14:58:41.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:40 smithi083 ceph-mon[32919]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:40 smithi083 ceph-mon[32919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:58:41.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:40 smithi083 ceph-mon[32919]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:41.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:41.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:41.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:41 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:42.152 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T14:58:42.158 INFO:tasks.cephadm:Deploying osd.7 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-25T14:58:42.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:42.185 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.186 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:41 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:42.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:42.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:42.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:41 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:42.404 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:58:42.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-mon[28090]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:58:42.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-mon[28090]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:58:42.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:42.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:42.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-mon[28090]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:43.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 14:58:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T14:58:42.958+0000 7f2b72a00700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T14:58:43.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 14:58:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T14:58:42.999+0000 7f2b68800700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:58:43.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:42 smithi083 ceph-mon[32919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:58:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:42 smithi083 ceph-mon[32919]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:58:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:43.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:42 smithi083 ceph-mon[32919]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:42 smithi102 ceph-mon[33005]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T14:58:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:42 smithi102 ceph-mon[33005]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:58:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:43.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:42 smithi102 ceph-mon[33005]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:44.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:43 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:58:44.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:43 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:58:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:43 smithi083 ceph-mon[32919]: from='osd.6 ' entity='osd.6' 2024-04-25T14:58:44.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:43 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:58:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:43 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:58:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:43 smithi019 ceph-mon[28090]: from='osd.6 ' entity='osd.6' 2024-04-25T14:58:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:43 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:58:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:43 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:58:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:44.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:43 smithi102 ceph-mon[33005]: from='osd.6 ' entity='osd.6' 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115138 s, 911 MB/s 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-04-25T14:58:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T14:58:45.004 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.005 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:44 smithi083 ceph-mon[32919]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:45.005 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:44 smithi083 ceph-mon[32919]: osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] boot 2024-04-25T14:58:45.005 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:44 smithi083 ceph-mon[32919]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:58:45.005 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:44 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-25T14:58:45.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:44 smithi019 ceph-mon[28090]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:44 smithi019 ceph-mon[28090]: osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] boot 2024-04-25T14:58:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:44 smithi019 ceph-mon[28090]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:58:45.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:44 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:44 smithi102 ceph-mon[33005]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:58:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:44 smithi102 ceph-mon[33005]: osd.6 [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] boot 2024-04-25T14:58:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:44 smithi102 ceph-mon[33005]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:58:45.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:44 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:58:45.312 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:58:46.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 -- 172.21.15.83:0/3472529449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d4103540 msgr2=0x7f12d4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 --2- 172.21.15.83:0/3472529449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d4103540 0x7f12d4109df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12c400b0d0 tx=0x7f12c400b3e0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 -- 172.21.15.83:0/3472529449 shutdown_connections 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 --2- 172.21.15.83:0/3472529449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d4103540 0x7f12d4109df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 --2- 172.21.15.83:0/3472529449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d41019a0 0x7f12d4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 --2- 172.21.15.83:0/3472529449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d4102ba0 0x7f12d4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 -- 172.21.15.83:0/3472529449 >> 172.21.15.83:0/3472529449 conn(0x7f12d40fcf40 msgr2=0x7f12d40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 -- 172.21.15.83:0/3472529449 shutdown_connections 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.873+0000 7f12dc025700 1 -- 172.21.15.83:0/3472529449 wait complete. 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 Processor -- start 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 -- start start 2024-04-25T14:58:46.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 0x7f12d4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d4103540 0x7f12d41a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d530 con 0x7f12d4103540 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d3b0 con 0x7f12d41019a0 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12dc025700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d6b0 con 0x7f12d4102ba0 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45738/0 (socket says 172.21.15.83:45738) 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 -- 172.21.15.83:0/1737972209 learned_addr learned my addr 172.21.15.83:0/1737972209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d95c0700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 0x7f12d4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 -- 172.21.15.83:0/1737972209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 msgr2=0x7f12d4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 0x7f12d4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.874+0000 7f12d9dc1700 1 -- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d4103540 msgr2=0x7f12d41a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d9dc1700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d4103540 0x7f12d41a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d9dc1700 1 -- 172.21.15.83:0/1737972209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d41a0950 con 0x7f12d41019a0 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d95c0700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 0x7f12d4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:58:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d9dc1700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00b500 tx=0x7f12cc00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:46.883 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:46 smithi083 ceph-mon[32919]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:58:46.883 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:46 smithi083 ceph-mon[32919]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:46.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc015020 con 0x7f12d41019a0 2024-04-25T14:58:46.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d41a0c40 con 0x7f12d41019a0 2024-04-25T14:58:46.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12cc011ce0 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc010d60 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.875+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d41a1180 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.876+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d4069730 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.878+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f12cc018020 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.878+0000 7f12d2ffd700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 0x7f12bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.878+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f12cc0cf4f0 con 0x7f12d41019a0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.879+0000 7f12d95c0700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 0x7f12bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.880+0000 7f12d95c0700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 0x7f12bc0a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c8007900 tx=0x7f12c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:46.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:46.882+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12cc01f020 con 0x7f12d41019a0 2024-04-25T14:58:47.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:58:47.087+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f12d40ff180 con 0x7f12bc0a3c40 2024-04-25T14:58:47.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:46 smithi019 ceph-mon[28090]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:58:47.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:46 smithi019 ceph-mon[28090]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:46 smithi102 ceph-mon[33005]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:58:47.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:46 smithi102 ceph-mon[33005]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:47.989 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:47.989 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:47.989 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi019 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 5698M 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:47.990 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi019 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 5698M 2024-04-25T14:58:48.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:48.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi019 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 5698M 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:48.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:58:49.014 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:48 smithi083 ceph-mon[32919]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:49.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:48 smithi019 ceph-mon[28090]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:49.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:48 smithi102 ceph-mon[33005]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:49 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/1365177732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:49 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:49 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]': finished 2024-04-25T14:58:50.217 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:49 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/1365177732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.217 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:49 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.217 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:49 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]': finished 2024-04-25T14:58:50.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:49 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/1365177732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:49 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]: dispatch 2024-04-25T14:58:50.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:49 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e2fed-875b-495d-aeae-bd3d93a14fa2"}]': finished 2024-04-25T14:58:51.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:50 smithi083 ceph-mon[32919]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:58:51.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:50 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:58:51.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:50 smithi083 ceph-mon[32919]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:51.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:50 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/4167624519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:51.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:50 smithi102 ceph-mon[33005]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:58:51.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:50 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:58:51.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:50 smithi102 ceph-mon[33005]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:51.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:50 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/4167624519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:51.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:50 smithi019 ceph-mon[28090]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:58:51.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:50 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:58:51.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:50 smithi019 ceph-mon[28090]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:51.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:50 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/4167624519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:53.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:52 smithi083 ceph-mon[32919]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:53.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:52 smithi102 ceph-mon[33005]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:53.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:52 smithi019 ceph-mon[28090]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:55.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:55 smithi083 ceph-mon[32919]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:55.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:55 smithi102 ceph-mon[33005]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:55.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:55 smithi019 ceph-mon[28090]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:57 smithi083 ceph-mon[32919]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:57 smithi102 ceph-mon[33005]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:57 smithi019 ceph-mon[28090]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:58.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:58:58.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:58.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:58 smithi083 ceph-mon[32919]: Deploying daemon osd.7 on smithi083 2024-04-25T14:58:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:58:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:58 smithi102 ceph-mon[33005]: Deploying daemon osd.7 on smithi083 2024-04-25T14:58:58.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:58:58.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:58.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:58 smithi019 ceph-mon[28090]: Deploying daemon osd.7 on smithi083 2024-04-25T14:58:59.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:58:59 smithi019 ceph-mon[28090]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:59.199 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:58:59 smithi083 ceph-mon[32919]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:58:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:58:59 smithi102 ceph-mon[33005]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:01 smithi083 ceph-mon[32919]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:01 smithi102 ceph-mon[33005]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:01.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:01 smithi019 ceph-mon[28090]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:03.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:03 smithi083 ceph-mon[32919]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:03 smithi102 ceph-mon[33005]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:03 smithi019 ceph-mon[28090]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:05.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:05 smithi083 ceph-mon[32919]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:05.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:05 smithi102 ceph-mon[33005]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:05 smithi019 ceph-mon[28090]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.325+0000 7f12d2ffd700 1 -- 172.21.15.83:0/1737972209 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12d40ff180 con 0x7f12bc0a3c40 2024-04-25T14:59:06.328 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 7 on host 'smithi083' 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 msgr2=0x7f12bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 0x7f12bc0a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c8007900 tx=0x7f12c8008040 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 msgr2=0x7f12d4198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00b500 tx=0x7f12cc00b8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 shutdown_connections 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f12bc0a3c40 0x7f12bc0a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d4103540 0x7f12d41a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12d41019a0 0x7f12d4198d00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 --2- 172.21.15.83:0/1737972209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d4102ba0 0x7f12d4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 >> 172.21.15.83:0/1737972209 conn(0x7f12d40fcf40 msgr2=0x7f12d40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:06.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 shutdown_connections 2024-04-25T14:59:06.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T14:59:06.328+0000 7f12dc025700 1 -- 172.21.15.83:0/1737972209 wait complete. 2024-04-25T14:59:06.942 DEBUG:teuthology.orchestra.run.smithi083:osd.7> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T14:59:06.944 INFO:tasks.cephadm:Deploying osd.8 on smithi102 with /dev/vg_nvme/lv_2... 2024-04-25T14:59:06.945 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:59:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.741 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:07.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:08.532 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:08 smithi102 ceph-mon[33005]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:59:08.532 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:08 smithi102 ceph-mon[33005]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:08.532 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:08 smithi102 ceph-mon[33005]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:59:08.533 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:08.533 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:08 smithi102 ceph-mon[33005]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:08.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:08 smithi019 ceph-mon[28090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:59:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:08 smithi019 ceph-mon[28090]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:08 smithi019 ceph-mon[28090]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:59:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:08.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:08 smithi019 ceph-mon[28090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:08.728 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:08 smithi083 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:59:08.728 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:08 smithi083 ceph-mon[32919]: from='osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:08.728 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:08 smithi083 ceph-mon[32919]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:59:08.728 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:08.728 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:08 smithi083 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T14:59:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:59:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:59:09.076 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-25T14:59:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-25T14:59:09.077 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128988 s, 813 MB/s 2024-04-25T14:59:09.077 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-25T14:59:09.340 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T14:59:09.225+0000 7f3c8f088700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T14:59:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: from='osd.7 ' entity='osd.7' 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:59:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: from='osd.7 ' entity='osd.7' 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:59:09.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.728 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-04-25T14:59:09.790 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T14:59:09.346+0000 7f3c86e73700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:59:09.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: from='osd.7 ' entity='osd.7' 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:59:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:09.966 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:59:10.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:10 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:59:10.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:10 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:59:10.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:10 smithi083 ceph-mon[32919]: osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] boot 2024-04-25T14:59:10.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:10 smithi083 ceph-mon[32919]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:59:10.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:10 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:10.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:10 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:59:10.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:10 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:59:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:10 smithi019 ceph-mon[28090]: osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] boot 2024-04-25T14:59:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:10 smithi019 ceph-mon[28090]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:59:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:10 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:10.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:10 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:59:10.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:10 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:59:10.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:10 smithi102 ceph-mon[33005]: osd.7 [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] boot 2024-04-25T14:59:10.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:10 smithi102 ceph-mon[33005]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:59:10.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:10 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:59:11.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 -- 172.21.15.102:0/3259606982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8102b90 msgr2=0x7f9aa8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/3259606982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8102b90 0x7f9aa8102ff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009990 tx=0x7f9a98009ca0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 -- 172.21.15.102:0/3259606982 shutdown_connections 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/3259606982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8103530 0x7f9aa8109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/3259606982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8101990 0x7f9aa8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/3259606982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8102b90 0x7f9aa8102ff0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.478+0000 7f9aafaa4700 1 -- 172.21.15.102:0/3259606982 >> 172.21.15.102:0/3259606982 conn(0x7f9aa80fcf70 msgr2=0x7f9aa80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- 172.21.15.102:0/3259606982 shutdown_connections 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- 172.21.15.102:0/3259606982 wait complete. 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 Processor -- start 2024-04-25T14:59:11.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- start start 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8101990 0x7f9aa806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8102b90 0x7f9aa819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa810d520 con 0x7f9aa8101990 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa810d3a0 con 0x7f9aa8102b90 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aafaa4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa810d6a0 con 0x7f9aa8103530 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.479+0000 7f9aae041700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35992/0 (socket says 172.21.15.102:35992) 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 -- 172.21.15.102:0/2405584866 learned_addr learned my addr 172.21.15.102:0/2405584866 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aad03f700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8102b90 0x7f9aa819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 -- 172.21.15.102:0/2405584866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8102b90 msgr2=0x7f9aa819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8102b90 0x7f9aa819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 -- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8101990 msgr2=0x7f9aa806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8101990 0x7f9aa806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 -- 172.21.15.102:0/2405584866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa81a0910 con 0x7f9aa8103530 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aae041700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000b970 tx=0x7f9aa000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0019070 con 0x7f9aa8103530 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa000cde0 con 0x7f9aa8103530 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa81a0c00 con 0x7f9aa8103530 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007370 con 0x7f9aa8103530 2024-04-25T14:59:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.480+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa81a1170 con 0x7f9aa8103530 2024-04-25T14:59:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.481+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8069730 con 0x7f9aa8103530 2024-04-25T14:59:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.482+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9aa001c070 con 0x7f9aa8103530 2024-04-25T14:59:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.483+0000 7f9a9effd700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 0x7f9a940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.483+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f9aa00cf500 con 0x7f9aa8103530 2024-04-25T14:59:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.483+0000 7f9aad840700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 0x7f9a940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.484+0000 7f9aad840700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 0x7f9a940a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9aa81029f0 tx=0x7f9aa4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.487+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aa0023070 con 0x7f9aa8103530 2024-04-25T14:59:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:11.696+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9aa8062e80 con 0x7f9a940a3c40 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi083 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:11 smithi102 ceph-mon[33005]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:59:11.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:11.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi083 2024-04-25T14:59:11.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:11.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:11.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:11 smithi019 ceph-mon[28090]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:59:12.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi083 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:12.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:12.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:12.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:12.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:11 smithi083 ceph-mon[32919]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:59:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:12 smithi102 ceph-mon[33005]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:12 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:12 smithi102 ceph-mon[33005]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:59:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:12 smithi019 ceph-mon[28090]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:12.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:12.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:12.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:12 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:12 smithi019 ceph-mon[28090]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:59:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:12 smithi083 ceph-mon[32919]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:12 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:12 smithi083 ceph-mon[32919]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:59:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:13 smithi102 ceph-mon[33005]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:13 smithi102 ceph-mon[33005]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:59:13.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:13 smithi019 ceph-mon[28090]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:13.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:13 smithi019 ceph-mon[28090]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:59:14.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:13 smithi083 ceph-mon[32919]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:14.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:13 smithi083 ceph-mon[32919]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3094077636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]': finished 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:15.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/1762320388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:59:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3094077636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]': finished 2024-04-25T14:59:15.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:59:15.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:15.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/1762320388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:15.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:15.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3094077636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]: dispatch 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b498ef1-2388-4402-9170-5c4a8e4cc29b"}]': finished 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:15.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/1762320388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:17.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:17 smithi083 ceph-mon[32919]: pgmap v208: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:17.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:17 smithi102 ceph-mon[33005]: pgmap v208: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:17.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:17 smithi019 ceph-mon[28090]: pgmap v208: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:19.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:19 smithi102 ceph-mon[33005]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:19.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:19 smithi083 ceph-mon[32919]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:19.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:19 smithi019 ceph-mon[28090]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:21.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:21 smithi083 ceph-mon[32919]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:21.546 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:21 smithi102 ceph-mon[33005]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:21.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:21 smithi019 ceph-mon[28090]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:22.497 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:59:22.497 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:22 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:22.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:59:22.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:22 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:22.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:59:22.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:22 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:23.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:23 smithi083 ceph-mon[32919]: Deploying daemon osd.8 on smithi102 2024-04-25T14:59:23.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:23 smithi083 ceph-mon[32919]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:23 smithi102 ceph-mon[33005]: Deploying daemon osd.8 on smithi102 2024-04-25T14:59:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:23 smithi102 ceph-mon[33005]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:23 smithi019 ceph-mon[28090]: Deploying daemon osd.8 on smithi102 2024-04-25T14:59:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:23 smithi019 ceph-mon[28090]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:25.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:25 smithi102 ceph-mon[33005]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:25.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:25 smithi083 ceph-mon[32919]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:25.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:25 smithi019 ceph-mon[28090]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:27.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:27 smithi083 ceph-mon[32919]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:27.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:27 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:27 smithi102 ceph-mon[33005]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:27 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:27 smithi019 ceph-mon[28090]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:27.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:27 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:29.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:29 smithi083 ceph-mon[32919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:29.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:29 smithi102 ceph-mon[33005]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:29.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:29 smithi019 ceph-mon[28090]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:30.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.634+0000 7f9a9effd700 1 -- 172.21.15.102:0/2405584866 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9aa8062e80 con 0x7f9a940a3c40 2024-04-25T14:59:30.635 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 8 on host 'smithi102' 2024-04-25T14:59:30.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 msgr2=0x7f9a940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:30.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 0x7f9a940a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9aa81029f0 tx=0x7f9aa4006d20 comp rx=0 tx=0).stop 2024-04-25T14:59:30.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 msgr2=0x7f9aa81a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:30.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000b970 tx=0x7f9aa000bc80 comp rx=0 tx=0).stop 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 shutdown_connections 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9aa8103530 0x7f9aa81a0040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a940a3c40 0x7f9a940a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8101990 0x7f9aa806bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 --2- 172.21.15.102:0/2405584866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9aa8102b90 0x7f9aa819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 >> 172.21.15.102:0/2405584866 conn(0x7f9aa80fcf70 msgr2=0x7f9aa8105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 shutdown_connections 2024-04-25T14:59:30.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T14:59:30.636+0000 7f9aafaa4700 1 -- 172.21.15.102:0/2405584866 wait complete. 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:30.888 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:30 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:31.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:30 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:31.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:31.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:30 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:31.250 DEBUG:teuthology.orchestra.run.smithi102:osd.8> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T14:59:31.252 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-04-25T14:59:31.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:59:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:59:32.131 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:31 smithi019 ceph-mon[28090]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:32.131 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:31 smithi019 ceph-mon[28090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:31 smithi083 ceph-mon[32919]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:31 smithi083 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:32.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:31 smithi102 ceph-mon[33005]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:32.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:31 smithi102 ceph-mon[33005]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:33.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:32 smithi019 ceph-mon[28090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:33.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:32 smithi083 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:59:33.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:59:33.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: from='osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:59:33.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T14:59:33.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:33.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:32 smithi102 ceph-mon[33005]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: from='osd.8 ' entity='osd.8' 2024-04-25T14:59:33.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:33.963 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T14:59:33.831+0000 7f5ef7d83700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: from='osd.8 ' entity='osd.8' 2024-04-25T14:59:34.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.237 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 14:59:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T14:59:33.986+0000 7f5eefb6e700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:59:34.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:59:34.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:59:34.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: from='osd.8 ' entity='osd.8' 2024-04-25T14:59:34.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036967 s, 284 MB/s 2024-04-25T14:59:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] boot 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi102 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:35.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] boot 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:35.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi102 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:35.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: osd.8 [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] boot 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi102 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.318 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:35.319 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T14:59:35.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-04-25T14:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:59:36.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:36 smithi083 ceph-mon[32919]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:59:36.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:36 smithi102 ceph-mon[33005]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:59:36.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:36 smithi019 ceph-mon[28090]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:59:37.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:37 smithi083 ceph-mon[32919]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:37 smithi083 ceph-mon[32919]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:59:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:37 smithi102 ceph-mon[33005]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:37 smithi102 ceph-mon[33005]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:59:37.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:37 smithi019 ceph-mon[28090]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:37 smithi019 ceph-mon[28090]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:59:38.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:38 smithi019 ceph-mon[28090]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:59:38.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:38 smithi083 ceph-mon[32919]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:59:38.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:38 smithi102 ceph-mon[33005]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:59:38.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 -- 172.21.15.19:0/4282451263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec104640 msgr2=0x7f6bec10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 --2- 172.21.15.19:0/4282451263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec104640 0x7f6bec10aef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009f90 tx=0x7f6be400b330 comp rx=0 tx=0).stop 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 -- 172.21.15.19:0/4282451263 shutdown_connections 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 --2- 172.21.15.19:0/4282451263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec104640 0x7f6bec10aef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 --2- 172.21.15.19:0/4282451263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec102aa0 0x7f6bec102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 --2- 172.21.15.19:0/4282451263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec103ca0 0x7f6bec104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 -- 172.21.15.19:0/4282451263 >> 172.21.15.19:0/4282451263 conn(0x7f6bec0fe040 msgr2=0x7f6bec100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 -- 172.21.15.19:0/4282451263 shutdown_connections 2024-04-25T14:59:38.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.619+0000 7f6bf4600700 1 -- 172.21.15.19:0/4282451263 wait complete. 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.620+0000 7f6bf4600700 1 Processor -- start 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.620+0000 7f6bf4600700 1 -- start start 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 0x7f6bec19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 0x7f6bec19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec104640 0x7f6bec1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec10e590 con 0x7f6bec103ca0 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec10e410 con 0x7f6bec104640 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf4600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec10e710 con 0x7f6bec102aa0 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 0x7f6bec19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 0x7f6bec19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48036/0 (socket says 172.21.15.19:48036) 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf1200700 1 -- 172.21.15.19:0/2259117220 learned_addr learned my addr 172.21.15.19:0/2259117220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf1c00700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 0x7f6bec19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.621+0000 7f6bf2600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec104640 0x7f6bec1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 -- 172.21.15.19:0/2259117220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec104640 msgr2=0x7f6bec1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec104640 0x7f6bec1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 -- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 msgr2=0x7f6bec19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 0x7f6bec19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 -- 172.21.15.19:0/2259117220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec1a2a50 con 0x7f6bec102aa0 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf1c00700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 0x7f6bec19ae40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6be000eb90 tx=0x7f6be000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:38.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000cc90 con 0x7f6bec102aa0 2024-04-25T14:59:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be000ce30 con 0x7f6bec102aa0 2024-04-25T14:59:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018880 con 0x7f6bec102aa0 2024-04-25T14:59:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec1a2d40 con 0x7f6bec102aa0 2024-04-25T14:59:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.622+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1a3170 con 0x7f6bec102aa0 2024-04-25T14:59:38.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.623+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bec1a33a0 con 0x7f6bec102aa0 2024-04-25T14:59:38.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.626+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6be001f020 con 0x7f6bec102aa0 2024-04-25T14:59:38.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.627+0000 7f6bea000700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 0x7f6bd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.627+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5829+0+0 (secure 0 0 0) 0x7f6be00cfcd0 con 0x7f6bec102aa0 2024-04-25T14:59:38.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.627+0000 7f6bf1200700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 0x7f6bd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.628+0000 7f6bf1200700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 0x7f6bd40a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009d60 tx=0x7f6bd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:38.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.630+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6be009cfc0 con 0x7f6bec102aa0 2024-04-25T14:59:38.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T14:59:38.870+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6bec1a3630 con 0x7f6bd40a3cc0 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:59:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:59:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:39.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:39.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 677 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:59:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:59:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:59:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:59:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:40.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:40 smithi083 ceph-mon[32919]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:40.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:40 smithi102 ceph-mon[33005]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:40.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:40 smithi019 ceph-mon[28090]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:41.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:41 smithi019 ceph-mon[28090]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:41.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:41 smithi083 ceph-mon[32919]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:41 smithi102 ceph-mon[33005]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:59:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:42 smithi019 ceph-mon[28090]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:59:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:42 smithi019 ceph-mon[28090]: Cluster is now healthy 2024-04-25T14:59:42.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:42 smithi083 ceph-mon[32919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:59:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:42 smithi083 ceph-mon[32919]: Cluster is now healthy 2024-04-25T14:59:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:42 smithi102 ceph-mon[33005]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:59:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:42 smithi102 ceph-mon[33005]: Cluster is now healthy 2024-04-25T14:59:43.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:43 smithi019 ceph-mon[28090]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:59:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:43 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]: dispatch 2024-04-25T14:59:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:43 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]': finished 2024-04-25T14:59:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:43 smithi019 ceph-mon[28090]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:59:43.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:59:43.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:43 smithi083 ceph-mon[32919]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:59:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:43 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]: dispatch 2024-04-25T14:59:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:43 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]': finished 2024-04-25T14:59:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:43 smithi083 ceph-mon[32919]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:59:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:59:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:43 smithi102 ceph-mon[33005]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:59:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:43 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]: dispatch 2024-04-25T14:59:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:43 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1952491769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7772cbd2-d2b5-4c1b-beaf-93651cd80d09"}]': finished 2024-04-25T14:59:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:43 smithi102 ceph-mon[33005]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:59:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:59:44.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:44 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/40872874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:44 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/40872874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:44.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/40872874' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:59:45.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:45 smithi083 ceph-mon[32919]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:59:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:45 smithi102 ceph-mon[33005]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:59:45.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:45 smithi019 ceph-mon[28090]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:59:47.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:47 smithi083 ceph-mon[32919]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:59:47.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:47 smithi102 ceph-mon[33005]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:59:47.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:47 smithi019 ceph-mon[28090]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:59:49.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:49 smithi019 ceph-mon[28090]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:59:49.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:49 smithi083 ceph-mon[32919]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:59:49.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:49 smithi102 ceph-mon[33005]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:59:51.586 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:51 smithi019 ceph-mon[28090]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:59:51.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:51 smithi083 ceph-mon[32919]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:59:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:51 smithi102 ceph-mon[33005]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:59:53.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:53 smithi083 ceph-mon[32919]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:53.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:59:53.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:53.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:53 smithi102 ceph-mon[33005]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:53.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:59:53.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:53.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:53 smithi019 ceph-mon[28090]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:53.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:59:53.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:54.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:54 smithi083 ceph-mon[32919]: Deploying daemon osd.9 on smithi019 2024-04-25T14:59:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:54 smithi102 ceph-mon[33005]: Deploying daemon osd.9 on smithi019 2024-04-25T14:59:54.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:54 smithi019 ceph-mon[28090]: Deploying daemon osd.9 on smithi019 2024-04-25T14:59:55.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:55 smithi019 ceph-mon[28090]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:55.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:55 smithi083 ceph-mon[32919]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:55.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:55 smithi102 ceph-mon[33005]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:57.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:57 smithi083 ceph-mon[32919]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:57.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:57 smithi019 ceph-mon[28090]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:57.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:57 smithi102 ceph-mon[33005]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:59.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 14:59:59 smithi019 ceph-mon[28090]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:59.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 14:59:59 smithi083 ceph-mon[32919]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:59:59.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 14:59:59 smithi102 ceph-mon[33005]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:00.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:00 smithi083 ceph-mon[32919]: overall HEALTH_OK 2024-04-25T15:00:00.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:00 smithi019 ceph-mon[28090]: overall HEALTH_OK 2024-04-25T15:00:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:00 smithi102 ceph-mon[33005]: overall HEALTH_OK 2024-04-25T15:00:01.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:01 smithi083 ceph-mon[32919]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:01.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:01 smithi102 ceph-mon[33005]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:01.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:01 smithi019 ceph-mon[28090]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:03 smithi019 ceph-mon[28090]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:03.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:03 smithi083 ceph-mon[32919]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:03 smithi102 ceph-mon[33005]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:05 smithi019 ceph-mon[28090]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:05.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:05 smithi083 ceph-mon[32919]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:05 smithi102 ceph-mon[33005]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:06.349 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:06 smithi019 ceph-mon[28090]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T15:00:06.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:06 smithi083 ceph-mon[32919]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T15:00:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:06 smithi102 ceph-mon[33005]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T15:00:07.174 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-04-25T15:00:07.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.172+0000 7f6bea000700 1 -- 172.21.15.19:0/2259117220 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6bec1a3630 con 0x7f6bd40a3cc0 2024-04-25T15:00:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 msgr2=0x7f6bd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 0x7f6bd40a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009d60 tx=0x7f6bd8009260 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 msgr2=0x7f6bec19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 0x7f6bec19ae40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6be000eb90 tx=0x7f6be000eea0 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 shutdown_connections 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bec104640 0x7f6bec1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bec102aa0 0x7f6bec19ae40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6bd40a3cc0 0x7f6bd40a6180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 --2- 172.21.15.19:0/2259117220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bec103ca0 0x7f6bec19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.175+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 >> 172.21.15.19:0/2259117220 conn(0x7f6bec0fe040 msgr2=0x7f6bec1094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.176+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 shutdown_connections 2024-04-25T15:00:07.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:00:07.176+0000 7f6bf4600700 1 -- 172.21.15.19:0/2259117220 wait complete. 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:07.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:07 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:07.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:07 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:07 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.093 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T15:00:08.100 INFO:tasks.cephadm:Deploying osd.10 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-25T15:00:08.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T15:00:08.348 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:00:08.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T15:00:08.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T15:00:08.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:08 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.469 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T15:00:08.469 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T15:00:08.469 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.470 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.470 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.470 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:08 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: from='osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T15:00:08.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T15:00:08.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:08.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:08.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:08 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:09.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T15:00:09.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T15:00:09.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:09.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:09.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] boot 2024-04-25T15:00:09.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T15:00:09.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:09 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:09.458 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] boot 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T15:00:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:09 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: osd.9 [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] boot 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T15:00:09.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:09 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:00:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T15:00:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T15:00:10.395 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T15:00:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T15:00:10.395 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011238 s, 933 MB/s 2024-04-25T15:00:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T15:00:11.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-25T15:00:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:00:11.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:11 smithi083 ceph-mon[32919]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:11.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:11 smithi083 ceph-mon[32919]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T15:00:11.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:11 smithi102 ceph-mon[33005]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:11.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:11 smithi102 ceph-mon[33005]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T15:00:11.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:11 smithi019 ceph-mon[28090]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:11.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:11 smithi019 ceph-mon[28090]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T15:00:12.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 -- 172.21.15.83:0/2081642879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 msgr2=0x7f4c74109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 --2- 172.21.15.83:0/2081642879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c74109e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b0d0 tx=0x7f4c6c00b3e0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 -- 172.21.15.83:0/2081642879 shutdown_connections 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 --2- 172.21.15.83:0/2081642879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c74109e10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 --2- 172.21.15.83:0/2081642879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 0x7f4c74101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 --2- 172.21.15.83:0/2081642879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74102bc0 0x7f4c74103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 -- 172.21.15.83:0/2081642879 >> 172.21.15.83:0/2081642879 conn(0x7f4c740fcfa0 msgr2=0x7f4c740ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 -- 172.21.15.83:0/2081642879 shutdown_connections 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.992+0000 7f4c79f10700 1 -- 172.21.15.83:0/2081642879 wait complete. 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 Processor -- start 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 -- start start 2024-04-25T15:00:12.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 0x7f4c74197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74102bc0 0x7f4c74198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410d4b0 con 0x7f4c74102bc0 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410d330 con 0x7f4c74103560 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.993+0000 7f4c79f10700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410d630 con 0x7f4c741019c0 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35160/0 (socket says 172.21.15.83:35160) 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 -- 172.21.15.83:0/423138316 learned_addr learned my addr 172.21.15.83:0/423138316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c737fe700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 0x7f4c74197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 -- 172.21.15.83:0/423138316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 msgr2=0x7f4c74197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 0x7f4c74197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 -- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74102bc0 msgr2=0x7f4c74198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74102bc0 0x7f4c74198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 -- 172.21.15.83:0/423138316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7419f720 con 0x7f4c74103560 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c73fff700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009040 tx=0x7f4c6c004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01f030 con 0x7f4c74103560 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c6c003a00 con 0x7f4c74103560 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0047b0 con 0x7f4c74103560 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7419f9b0 con 0x7f4c74103560 2024-04-25T15:00:12.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.994+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7419fee0 con 0x7f4c74103560 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.995+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c74069730 con 0x7f4c74103560 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.996+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c6c003bb0 con 0x7f4c74103560 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.997+0000 7f4c70ff9700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 0x7f4c5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.997+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7f4c6c00f020 con 0x7f4c74103560 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.997+0000 7f4c737fe700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 0x7f4c5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:12.998+0000 7f4c737fe700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 0x7f4c5c0a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c74102a20 tx=0x7f4c6400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:13.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:13.002+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c6c029020 con 0x7f4c74103560 2024-04-25T15:00:13.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:13.208+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c74062e80 con 0x7f4c5c0a3cc0 2024-04-25T15:00:13.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:13 smithi083 ceph-mon[32919]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:13 smithi102 ceph-mon[33005]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:13.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:13 smithi019 ceph-mon[28090]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:14.456 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi019 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.457 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.458 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi019 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.559 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi019 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:14.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:15 smithi083 ceph-mon[32919]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:16.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:15 smithi102 ceph-mon[33005]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:16.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:15 smithi019 ceph-mon[28090]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/166916153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]': finished 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.83:0/1997581502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/166916153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]': finished 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.83:0/1997581502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:17.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/166916153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]: dispatch 2024-04-25T15:00:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "924537e9-24c0-45be-8ae2-f5623c73902b"}]': finished 2024-04-25T15:00:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T15:00:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:17.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.83:0/1997581502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:18.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:17 smithi083 ceph-mon[32919]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:18.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:17 smithi102 ceph-mon[33005]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:18.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:17 smithi019 ceph-mon[28090]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:19 smithi083 ceph-mon[32919]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:20.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:19 smithi102 ceph-mon[33005]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:20.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:19 smithi019 ceph-mon[28090]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:22.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:21 smithi083 ceph-mon[32919]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:22.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:21 smithi102 ceph-mon[33005]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:22.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:21 smithi019 ceph-mon[28090]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:24.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:23 smithi083 ceph-mon[32919]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:24.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:23 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T15:00:24.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:23 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:24.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:23 smithi102 ceph-mon[33005]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:24.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:23 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T15:00:24.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:23 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:24.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:23 smithi019 ceph-mon[28090]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:24.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:23 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T15:00:24.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:23 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:25.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:24 smithi083 ceph-mon[32919]: Deploying daemon osd.10 on smithi083 2024-04-25T15:00:25.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:24 smithi102 ceph-mon[33005]: Deploying daemon osd.10 on smithi083 2024-04-25T15:00:25.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:24 smithi019 ceph-mon[28090]: Deploying daemon osd.10 on smithi083 2024-04-25T15:00:26.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:25 smithi083 ceph-mon[32919]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:26.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:25 smithi102 ceph-mon[33005]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:26.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:25 smithi019 ceph-mon[28090]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:28.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:27 smithi083 ceph-mon[32919]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:28.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:27 smithi102 ceph-mon[33005]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:28.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:27 smithi019 ceph-mon[28090]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:30.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:29 smithi083 ceph-mon[32919]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:30.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:30.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:29 smithi102 ceph-mon[33005]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:30.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:30.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:29 smithi019 ceph-mon[28090]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:30.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:30.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:32.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:31 smithi083 ceph-mon[32919]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:32.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:31 smithi102 ceph-mon[33005]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:32.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:31 smithi019 ceph-mon[28090]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:33.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.248+0000 7f4c70ff9700 1 -- 172.21.15.83:0/423138316 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4c74062e80 con 0x7f4c5c0a3cc0 2024-04-25T15:00:33.249 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 10 on host 'smithi083' 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 msgr2=0x7f4c5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 0x7f4c5c0a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c74102a20 tx=0x7f4c6400b410 comp rx=0 tx=0).stop 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 msgr2=0x7f4c7419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009040 tx=0x7f4c6c004d50 comp rx=0 tx=0).stop 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 shutdown_connections 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74103560 0x7f4c7419f080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c741019c0 0x7f4c74197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c5c0a3cc0 0x7f4c5c0a6180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 --2- 172.21.15.83:0/423138316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74102bc0 0x7f4c74198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 >> 172.21.15.83:0/423138316 conn(0x7f4c740fcfa0 msgr2=0x7f4c74105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 shutdown_connections 2024-04-25T15:00:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T15:00:33.250+0000 7f4c79f10700 1 -- 172.21.15.83:0/423138316 wait complete. 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:33 smithi083 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:33 smithi102 ceph-mon[33005]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:33.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:33 smithi019 ceph-mon[28090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T15:00:33.918 DEBUG:teuthology.orchestra.run.smithi083:osd.10> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T15:00:33.920 INFO:tasks.cephadm:Deploying osd.11 on smithi102 with /dev/vg_nvme/lv_1... 2024-04-25T15:00:33.920 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T15:00:34.163 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:35.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:35.404 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:00:35.242+0000 7f280460f700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T15:00:35.404 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:00:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:00:35.255+0000 7f2800402700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:00:35.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:35.496 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T15:00:35.496 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:35.496 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T15:00:35.496 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:35.496 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:35 smithi102 ceph-mon[33005]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: from='osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:35.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:35 smithi019 ceph-mon[28090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T15:00:36.209 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T15:00:36.210 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T15:00:36.210 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-25T15:00:36.210 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-25T15:00:36.210 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114444 s, 916 MB/s 2024-04-25T15:00:36.210 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-25T15:00:36.506 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:36 smithi083 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T15:00:36.506 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:36 smithi083 ceph-mon[32919]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T15:00:36.506 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:36 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.506 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:36 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:36 smithi102 ceph-mon[33005]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T15:00:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:36 smithi102 ceph-mon[33005]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T15:00:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:36 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:36 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:36 smithi019 ceph-mon[28090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T15:00:36.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:36 smithi019 ceph-mon[28090]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T15:00:36.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:36 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:36 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:36.870 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-04-25T15:00:37.116 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:00:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] boot 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:37.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:37.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T15:00:37.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] boot 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:37.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: osd.10 [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] boot 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T15:00:37.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:37.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:38.451 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:38 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi083 2024-04-25T15:00:38.451 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:38 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T15:00:38.451 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:38 smithi102 ceph-mon[33005]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T15:00:38.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:38 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi083 2024-04-25T15:00:38.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:38 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T15:00:38.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:38 smithi083 ceph-mon[32919]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T15:00:38.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:38 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi083 2024-04-25T15:00:38.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:38 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T15:00:38.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:38 smithi019 ceph-mon[28090]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- 172.21.15.102:0/640722550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614102aa0 msgr2=0x7f9614102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- 172.21.15.102:0/640722550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614102aa0 0x7f9614102ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9608009990 tx=0x7f9608009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- 172.21.15.102:0/640722550 shutdown_connections 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- 172.21.15.102:0/640722550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 0x7f961410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- 172.21.15.102:0/640722550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614102aa0 0x7f9614102ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- 172.21.15.102:0/640722550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9614103ca0 0x7f9614104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- 172.21.15.102:0/640722550 >> 172.21.15.102:0/640722550 conn(0x7f96140fe040 msgr2=0x7f9614100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- 172.21.15.102:0/640722550 shutdown_connections 2024-04-25T15:00:38.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- 172.21.15.102:0/640722550 wait complete. 2024-04-25T15:00:38.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 Processor -- start 2024-04-25T15:00:38.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- start start 2024-04-25T15:00:38.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9614102aa0 0x7f9614198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:38.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 0x7f96141a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f961410e610 con 0x7f9614102aa0 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f961410e490 con 0x7f9614104640 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.645+0000 7f961ad2a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f961410e790 con 0x7f9614103ca0 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:39856/0 (socket says 172.21.15.102:39856) 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 -- 172.21.15.102:0/2784730866 learned_addr learned my addr 172.21.15.102:0/2784730866 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f96192c7700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 0x7f96141a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 -- 172.21.15.102:0/2784730866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 msgr2=0x7f96141a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 0x7f96141a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 -- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9614102aa0 msgr2=0x7f9614198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9614102aa0 0x7f9614198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 -- 172.21.15.102:0/2784730866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96141a0990 con 0x7f9614103ca0 2024-04-25T15:00:38.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9613fff700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f960000b560 tx=0x7f960000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600015020 con 0x7f9614103ca0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9600011ce0 con 0x7f9614103ca0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96141a0c80 con 0x7f9614103ca0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.646+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600010da0 con 0x7f9614103ca0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.647+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96141a11c0 con 0x7f9614103ca0 2024-04-25T15:00:38.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.647+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9614069730 con 0x7f9614103ca0 2024-04-25T15:00:38.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.648+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9600018020 con 0x7f9614103ca0 2024-04-25T15:00:38.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.649+0000 7f9611ffb700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 0x7f96040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:38.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.649+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7f96000cf5c0 con 0x7f9614103ca0 2024-04-25T15:00:38.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.649+0000 7f9618ac6700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 0x7f96040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:38.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.650+0000 7f9618ac6700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 0x7f96040a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f960800b5c0 tx=0x7f9608003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:38.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.653+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f960001f020 con 0x7f9614103ca0 2024-04-25T15:00:38.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:38.862+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9614062e80 con 0x7f96040a3c40 2024-04-25T15:00:39.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:39 smithi083 ceph-mon[32919]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:39.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:39 smithi083 ceph-mon[32919]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T15:00:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:39 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:39 smithi102 ceph-mon[33005]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:39 smithi102 ceph-mon[33005]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T15:00:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:39.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:39.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:39 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:39 smithi019 ceph-mon[28090]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:39 smithi019 ceph-mon[28090]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T15:00:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:00:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:00:39.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:39 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:40.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:40 smithi019 ceph-mon[28090]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:40.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:40 smithi019 ceph-mon[28090]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T15:00:40.449 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:40 smithi102 ceph-mon[33005]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:40.449 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:40 smithi102 ceph-mon[33005]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T15:00:40.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:40 smithi083 ceph-mon[32919]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:00:40.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:40 smithi083 ceph-mon[32919]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T15:00:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:41 smithi102 ceph-mon[33005]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:41 smithi102 ceph-mon[33005]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T15:00:41.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:41 smithi019 ceph-mon[28090]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:41.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:41 smithi019 ceph-mon[28090]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T15:00:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:41 smithi083 ceph-mon[32919]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:41 smithi083 ceph-mon[32919]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T15:00:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/3055700810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]': finished 2024-04-25T15:00:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T15:00:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:00:42.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:42 smithi019 ceph-mon[28090]: from='client.? 172.21.15.102:0/448636420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:42.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/3055700810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]': finished 2024-04-25T15:00:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T15:00:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:00:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:42 smithi083 ceph-mon[32919]: from='client.? 172.21.15.102:0/448636420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/3055700810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]: dispatch 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6578f249-8ff7-4cd6-b80f-9b7b873d0bc2"}]': finished 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:00:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:42 smithi102 ceph-mon[33005]: from='client.? 172.21.15.102:0/448636420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:00:44.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:43 smithi083 ceph-mon[32919]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T15:00:44.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:43 smithi102 ceph-mon[33005]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T15:00:44.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:43 smithi019 ceph-mon[28090]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T15:00:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:45 smithi102 ceph-mon[33005]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T15:00:46.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:45 smithi083 ceph-mon[32919]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T15:00:46.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:45 smithi019 ceph-mon[28090]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T15:00:47.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:47 smithi102 ceph-mon[33005]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T15:00:48.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:47 smithi083 ceph-mon[32919]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T15:00:48.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:47 smithi019 ceph-mon[28090]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T15:00:49.906 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:49 smithi102 ceph-mon[33005]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T15:00:49.906 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T15:00:49.906 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:49 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:50.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:49 smithi083 ceph-mon[32919]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T15:00:50.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T15:00:50.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:49 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:49 smithi019 ceph-mon[28090]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T15:00:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T15:00:50.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:49 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:51.032 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:50 smithi083 ceph-mon[32919]: Deploying daemon osd.11 on smithi102 2024-04-25T15:00:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:50 smithi102 ceph-mon[33005]: Deploying daemon osd.11 on smithi102 2024-04-25T15:00:51.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:50 smithi019 ceph-mon[28090]: Deploying daemon osd.11 on smithi102 2024-04-25T15:00:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:51 smithi102 ceph-mon[33005]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T15:00:52.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:51 smithi083 ceph-mon[32919]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T15:00:52.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:51 smithi019 ceph-mon[28090]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T15:00:53.734 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:53 smithi102 ceph-mon[33005]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:54.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:53 smithi083 ceph-mon[32919]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:54.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:53 smithi019 ceph-mon[28090]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:56.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:55 smithi083 ceph-mon[32919]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:55 smithi102 ceph-mon[33005]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:55 smithi019 ceph-mon[28090]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:57.924 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:57 smithi019 ceph-mon[28090]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:57.924 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:57.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:57.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:57.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:58.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:57 smithi083 ceph-mon[32919]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:58.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:58.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:57 smithi102 ceph-mon[33005]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:00:58.501 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 11 on host 'smithi102' 2024-04-25T15:00:58.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.500+0000 7f9611ffb700 1 -- 172.21.15.102:0/2784730866 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9614062e80 con 0x7f96040a3c40 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 msgr2=0x7f96040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 0x7f96040a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f960800b5c0 tx=0x7f9608003860 comp rx=0 tx=0).stop 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 msgr2=0x7f9614199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f960000b560 tx=0x7f960000b920 comp rx=0 tx=0).stop 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 shutdown_connections 2024-04-25T15:00:58.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9614104640 0x7f96141a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f96040a3c40 0x7f96040a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9614102aa0 0x7f9614198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 --2- 172.21.15.102:0/2784730866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9614103ca0 0x7f9614199270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 >> 172.21.15.102:0/2784730866 conn(0x7f96140fe040 msgr2=0x7f96141070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 shutdown_connections 2024-04-25T15:00:58.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:00:58.502+0000 7f961ad2a700 1 -- 172.21.15.102:0/2784730866 wait complete. 2024-04-25T15:00:58.680 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:58.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:58.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:58.681 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:58 smithi102 ceph-mon[33005]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:59.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:58 smithi083 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:59.113 DEBUG:teuthology.orchestra.run.smithi102:osd.11> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T15:00:59.115 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T15:00:59.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:59.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:58 smithi019 ceph-mon[28090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T15:00:59.565 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:59.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T15:00:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:00:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T15:00:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:00:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:00:59 smithi019 ceph-mon[28090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:00:59.957 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:00:59.958 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T15:00:59.958 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:00:59.958 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T15:00:59.958 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:00:59.958 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:00:59 smithi102 ceph-mon[33005]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: from='osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:00.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:00:59 smithi083 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.731 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:01 smithi019 ceph-mon[28090]: from='osd.11 ' entity='osd.11' 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:01 smithi083 ceph-mon[32919]: from='osd.11 ' entity='osd.11' 2024-04-25T15:01:01.804 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:01.804 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-25T15:01:01.804 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T15:01:01.805 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.805 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:01.805 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-mon[33005]: from='osd.11 ' entity='osd.11' 2024-04-25T15:01:01.805 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:01:01.387+0000 7f16db2c5700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T15:01:01.805 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:01:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:01:01.558+0000 7f16d30b0700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:01:02.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 -- 172.21.15.19:0/3886119153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe288102ba0 msgr2=0x7fe288103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3886119153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe288102ba0 0x7fe288103000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe2740098f0 tx=0x7fe274009c00 comp rx=0 tx=0).stop 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 -- 172.21.15.19:0/3886119153 shutdown_connections 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3886119153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288103540 0x7fe288109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3886119153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2881019a0 0x7fe288101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3886119153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe288102ba0 0x7fe288103000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 -- 172.21.15.19:0/3886119153 >> 172.21.15.19:0/3886119153 conn(0x7fe2880fcf40 msgr2=0x7fe2880ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 -- 172.21.15.19:0/3886119153 shutdown_connections 2024-04-25T15:01:02.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.249+0000 7fe28fe00700 1 -- 172.21.15.19:0/3886119153 wait complete. 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.250+0000 7fe28fe00700 1 Processor -- start 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 -- start start 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 0x7fe2881058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288103540 0x7fe2881044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe28810d490 con 0x7fe288103540 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe28810d310 con 0x7fe288102ba0 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.251+0000 7fe28fe00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe28810d610 con 0x7fe2881019a0 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288103540 0x7fe2881044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:57616/0 (socket says 172.21.15.19:57616) 2024-04-25T15:01:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 -- 172.21.15.19:0/3021123818 learned_addr learned my addr 172.21.15.19:0/3021123818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 0x7fe2881058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 -- 172.21.15.19:0/3021123818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 msgr2=0x7fe2881058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 0x7fe2881058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 -- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288103540 msgr2=0x7fe2881044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288103540 0x7fe2881044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 -- 172.21.15.19:0/3021123818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288104d90 con 0x7fe288102ba0 2024-04-25T15:01:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28d400700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 0x7fe2881058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.252+0000 7fe28ca00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe274010fa0 tx=0x7fe274010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.253+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274027070 con 0x7fe288102ba0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.253+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27400b560 con 0x7fe288102ba0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.253+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274004930 con 0x7fe288102ba0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.253+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a2de0 con 0x7fe288102ba0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.253+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881a31d0 con 0x7fe288102ba0 2024-04-25T15:01:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.254+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe288069730 con 0x7fe288102ba0 2024-04-25T15:01:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.256+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe27402a070 con 0x7fe288102ba0 2024-04-25T15:01:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.257+0000 7fe286000700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 0x7fe2700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.258+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7fe2740d0870 con 0x7fe288102ba0 2024-04-25T15:01:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.258+0000 7fe28d400700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 0x7fe2700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.258+0000 7fe28d400700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 0x7fe2700a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe288102a00 tx=0x7fe27c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.262+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe27405eff0 con 0x7fe288102ba0 2024-04-25T15:01:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.503+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe2881a3b70 con 0x7fe288102ba0 2024-04-25T15:01:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.504+0000 7fe286000700 1 -- 172.21.15.19:0/3021123818 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7fe274013030 con 0x7fe288102ba0 2024-04-25T15:01:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:02.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 msgr2=0x7fe2700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:02.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 0x7fe2700a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe288102a00 tx=0x7fe27c009310 comp rx=0 tx=0).stop 2024-04-25T15:01:02.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 msgr2=0x7fe288103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:02.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe274010fa0 tx=0x7fe274010fd0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 shutdown_connections 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe2700a3cc0 0x7fe2700a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288103540 0x7fe2881044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2881019a0 0x7fe2881058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 --2- 172.21.15.19:0/3021123818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe288102ba0 0x7fe288103f80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 >> 172.21.15.19:0/3021123818 conn(0x7fe2880fcf40 msgr2=0x7fe28806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 shutdown_connections 2024-04-25T15:01:02.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:02.508+0000 7fe28fe00700 1 -- 172.21.15.19:0/3021123818 wait complete. 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: purged_snaps scrub starts 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: purged_snaps scrub ok 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:02.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: purged_snaps scrub starts 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: purged_snaps scrub ok 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:02.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: purged_snaps scrub starts 2024-04-25T15:01:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: purged_snaps scrub ok 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:02.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:03.311 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714057262,"num_in_osds":12,"osd_in_since":1714057241,"num_remapped_pgs":0} 2024-04-25T15:01:03.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:01:03.772 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:03.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:03.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: Detected new or changed devices on smithi102 2024-04-25T15:01:03.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-25T15:01:03.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] boot 2024-04-25T15:01:03.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T15:01:03.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:03 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3021123818' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: Detected new or changed devices on smithi102 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] boot 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:03 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3021123818' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:01:03.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: Detected new or changed devices on smithi102 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: osd.11 [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] boot 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:01:03.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:03 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3021123818' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:01:04.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:04 smithi083 ceph-mon[32919]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T15:01:04.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:04 smithi102 ceph-mon[33005]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T15:01:04.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:04 smithi019 ceph-mon[28090]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T15:01:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:05 smithi019 ceph-mon[28090]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:05.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:05 smithi083 ceph-mon[32919]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:05 smithi102 ceph-mon[33005]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 -- 172.21.15.19:0/1640213101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 msgr2=0x7fbb6010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 --2- 172.21.15.19:0/1640213101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb6010dcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0098f0 tx=0x7fbb4c009c00 comp rx=0 tx=0).stop 2024-04-25T15:01:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 -- 172.21.15.19:0/1640213101 shutdown_connections 2024-04-25T15:01:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 --2- 172.21.15.19:0/1640213101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb60115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 --2- 172.21.15.19:0/1640213101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 --2- 172.21.15.19:0/1640213101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb6010dcd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 -- 172.21.15.19:0/1640213101 >> 172.21.15.19:0/1640213101 conn(0x7fbb60076210 msgr2=0x7fbb60107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 -- 172.21.15.19:0/1640213101 shutdown_connections 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.275+0000 7fbb66c00700 1 -- 172.21.15.19:0/1640213101 wait complete. 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.276+0000 7fbb66c00700 1 Processor -- start 2024-04-25T15:01:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.276+0000 7fbb66c00700 1 -- start start 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb601a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb6019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60119270 con 0x7fbb601092d0 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601190f0 con 0x7fbb60109c80 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb66c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601193f0 con 0x7fbb6010e3b0 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb5fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb5fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36860/0 (socket says 172.21.15.19:36860) 2024-04-25T15:01:06.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5fe00700 1 -- 172.21.15.19:0/200932217 learned_addr learned my addr 172.21.15.19:0/200932217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.277+0000 7fbb5f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb601a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb64c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb6019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 -- 172.21.15.19:0/200932217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 msgr2=0x7fbb6019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb6019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 -- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 msgr2=0x7fbb601a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 -- 172.21.15.19:0/200932217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6019fd30 con 0x7fbb60109c80 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb64c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb6019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5fe00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.278+0000 7fbb5f400700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb601a5950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c00fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.279+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c005140 con 0x7fbb60109c80 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.279+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6019ffc0 con 0x7fbb60109c80 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.279+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601ac2e0 con 0x7fbb60109c80 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.279+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c00b560 con 0x7fbb60109c80 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.279+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c003a40 con 0x7fbb60109c80 2024-04-25T15:01:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.280+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb40005330 con 0x7fbb60109c80 2024-04-25T15:01:06.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.282+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb4c003be0 con 0x7fbb60109c80 2024-04-25T15:01:06.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.283+0000 7fbb5cc00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 0x7fbb440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.284+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fbb4c0d0d40 con 0x7fbb60109c80 2024-04-25T15:01:06.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.284+0000 7fbb5fe00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 0x7fbb440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.284+0000 7fbb5fe00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 0x7fbb440a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb50011550 tx=0x7fbb50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.288+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb4c013030 con 0x7fbb60109c80 2024-04-25T15:01:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.523+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb40005620 con 0x7fbb60109c80 2024-04-25T15:01:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.525+0000 7fbb5cc00700 1 -- 172.21.15.19:0/200932217 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18385 (secure 0 0 0) 0x7fbb4c09dbb0 con 0x7fbb60109c80 2024-04-25T15:01:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:06.527 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":84,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:01:03.497686+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.528+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 msgr2=0x7fbb440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.528+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 0x7fbb440a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb50011550 tx=0x7fbb50009260 comp rx=0 tx=0).stop 2024-04-25T15:01:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.528+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 msgr2=0x7fbb601a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.528+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb601a5950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c00fa20 comp rx=0 tx=0).stop 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 shutdown_connections 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb6010e3b0 0x7fbb6019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbb440a3cc0 0x7fbb440a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb601092d0 0x7fbb601a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 --2- 172.21.15.19:0/200932217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60109c80 0x7fbb601a5950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 >> 172.21.15.19:0/200932217 conn(0x7fbb60076210 msgr2=0x7fbb601086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 shutdown_connections 2024-04-25T15:01:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:06.529+0000 7fbb66c00700 1 -- 172.21.15.19:0/200932217 wait complete. 2024-04-25T15:01:07.394 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:07.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T15:01:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:07 smithi019 ceph-mon[28090]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:07 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/200932217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:07.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:07 smithi083 ceph-mon[32919]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:07.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:07 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/200932217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:07 smithi102 ceph-mon[33005]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:07 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/200932217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:07.882 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:09.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:09 smithi019 ceph-mon[28090]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:09.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:09 smithi083 ceph-mon[32919]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:09.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:09 smithi102 ceph-mon[33005]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 -- 172.21.15.19:0/3399693107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc102ad0 msgr2=0x7f46cc102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 --2- 172.21.15.19:0/3399693107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc102ad0 0x7f46cc102f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009990 tx=0x7f46b8009ca0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 -- 172.21.15.19:0/3399693107 shutdown_connections 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 --2- 172.21.15.19:0/3399693107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103490 0x7f46cc10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 --2- 172.21.15.19:0/3399693107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc108aa0 0x7f46cc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 --2- 172.21.15.19:0/3399693107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc102ad0 0x7f46cc102f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.068+0000 7f46d1200700 1 -- 172.21.15.19:0/3399693107 >> 172.21.15.19:0/3399693107 conn(0x7f46cc079c30 msgr2=0x7f46cc07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.069+0000 7f46d1200700 1 -- 172.21.15.19:0/3399693107 shutdown_connections 2024-04-25T15:01:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.069+0000 7f46d1200700 1 -- 172.21.15.19:0/3399693107 wait complete. 2024-04-25T15:01:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 Processor -- start 2024-04-25T15:01:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 -- start start 2024-04-25T15:01:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc102ad0 0x7f46cc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 0x7f46cc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc112cd0 con 0x7f46cc108aa0 2024-04-25T15:01:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc112b50 con 0x7f46cc102ad0 2024-04-25T15:01:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.070+0000 7f46d1200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc112e50 con 0x7f46cc103490 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:35378/0 (socket says 172.21.15.19:35378) 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc102ad0 0x7f46cc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 -- 172.21.15.19:0/4081279921 learned_addr learned my addr 172.21.15.19:0/4081279921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46ca800700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 0x7f46cc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 -- 172.21.15.19:0/4081279921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc102ad0 msgr2=0x7f46cc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc102ad0 0x7f46cc074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 -- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 msgr2=0x7f46cc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 0x7f46cc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 -- 172.21.15.19:0/4081279921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc1a7770 con 0x7f46cc103490 2024-04-25T15:01:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46ca800700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 0x7f46cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.071+0000 7f46c9400700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46b80057c0 tx=0x7f46b80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.072+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800fad0 con 0x7f46cc103490 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.072+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46b800fc70 con 0x7f46cc103490 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.072+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1a7a00 con 0x7f46cc103490 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.072+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8017650 con 0x7f46cc103490 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.072+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a7e50 con 0x7f46cc103490 2024-04-25T15:01:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.073+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc069730 con 0x7f46cc103490 2024-04-25T15:01:10.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.075+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46b8009d10 con 0x7f46cc103490 2024-04-25T15:01:10.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.077+0000 7f46c2a00700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 0x7f46b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:10.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.077+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f46b80d0f80 con 0x7f46cc103490 2024-04-25T15:01:10.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.077+0000 7f46c9e00700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 0x7f46b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:10.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.078+0000 7f46c9e00700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 0x7f46b00a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005950 tx=0x7f46bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:10.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.085+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46b8013070 con 0x7f46cc103490 2024-04-25T15:01:10.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.315+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46cc02e140 con 0x7f46cc103490 2024-04-25T15:01:10.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.315+0000 7f46c2a00700 1 -- 172.21.15.19:0/4081279921 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f46b809ddf0 con 0x7f46cc103490 2024-04-25T15:01:10.317 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.318+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 msgr2=0x7f46b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.318+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 0x7f46b00a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005950 tx=0x7f46bc00b410 comp rx=0 tx=0).stop 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.318+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 msgr2=0x7f46cc075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.318+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46b80057c0 tx=0x7f46b80057f0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 shutdown_connections 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46b00a3cc0 0x7f46b00a6180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc108aa0 0x7f46cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46cc102ad0 0x7f46cc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 --2- 172.21.15.19:0/4081279921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46cc103490 0x7f46cc075280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 >> 172.21.15.19:0/4081279921 conn(0x7f46cc079c30 msgr2=0x7f46cc100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 shutdown_connections 2024-04-25T15:01:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:10.319+0000 7f46d1200700 1 -- 172.21.15.19:0/4081279921 wait complete. 2024-04-25T15:01:10.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:10 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4081279921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:10.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:10 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4081279921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:10.806 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:10 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4081279921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:11.152 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.b on smithi083 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.e on smithi083 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.h on smithi083 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.k on smithi083 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.c on smithi102 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.f on smithi102 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.i on smithi102 2024-04-25T15:01:11.153 INFO:tasks.cephadm:Adding mds.l on smithi102 2024-04-25T15:01:11.153 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l' 2024-04-25T15:01:11.387 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:01:11.761 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:11 smithi102 ceph-mon[33005]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:11.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:11 smithi083 ceph-mon[32919]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:11.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:11 smithi019 ceph-mon[28090]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.042+0000 7fe798abd700 1 -- 172.21.15.102:0/3018130969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 msgr2=0x7fe7941033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.042+0000 7fe798abd700 1 --2- 172.21.15.102:0/3018130969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941033a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7840099c0 tx=0x7fe784009cd0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 -- 172.21.15.102:0/3018130969 shutdown_connections 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 --2- 172.21.15.102:0/3018130969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 0x7fe794109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 --2- 172.21.15.102:0/3018130969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941033a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 --2- 172.21.15.102:0/3018130969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe794101cc0 0x7fe794102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 -- 172.21.15.102:0/3018130969 >> 172.21.15.102:0/3018130969 conn(0x7fe7940fc710 msgr2=0x7fe7940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 -- 172.21.15.102:0/3018130969 shutdown_connections 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 -- 172.21.15.102:0/3018130969 wait complete. 2024-04-25T15:01:13.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 Processor -- start 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.043+0000 7fe798abd700 1 -- start start 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe794101cc0 0x7fe794198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 0x7fe7941991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410d4e0 con 0x7fe794101cc0 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410d360 con 0x7fe794102660 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410d660 con 0x7fe794102f80 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46534/0 (socket says 172.21.15.102:46534) 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 -- 172.21.15.102:0/2080515634 learned_addr learned my addr 172.21.15.102:0/2080515634 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe791d9b700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 0x7fe7941991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 -- 172.21.15.102:0/2080515634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 msgr2=0x7fe7941991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 0x7fe7941991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 -- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe794101cc0 msgr2=0x7fe794198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe794101cc0 0x7fe794198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 -- 172.21.15.102:0/2080515634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7941a08c0 con 0x7fe794102f80 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe792d9d700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ada0 tx=0x7fe78c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c017020 con 0x7fe794102f80 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c00ce30 con 0x7fe794102f80 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941a0bb0 con 0x7fe794102f80 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c005350 con 0x7fe794102f80 2024-04-25T15:01:13.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.044+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941a10f0 con 0x7fe794102f80 2024-04-25T15:01:13.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.045+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe794069730 con 0x7fe794102f80 2024-04-25T15:01:13.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.047+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe78c00c980 con 0x7fe794102f80 2024-04-25T15:01:13.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.047+0000 7fe7837fe700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 0x7fe77c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.048+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe78c0cf630 con 0x7fe794102f80 2024-04-25T15:01:13.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.048+0000 7fe79259c700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 0x7fe77c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.049+0000 7fe79259c700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 0x7fe77c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe784003790 tx=0x7fe784009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:13.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.052+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe78c021020 con 0x7fe794102f80 2024-04-25T15:01:13.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.257+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}) -- 0x7fe7940fea00 con 0x7fe77c0a3c40 2024-04-25T15:01:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.274+0000 7fe7837fe700 1 -- 172.21.15.102:0/2080515634 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe7940fea00 con 0x7fe77c0a3c40 2024-04-25T15:01:13.276 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mds.all update... 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 msgr2=0x7fe77c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 0x7fe77c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe784003790 tx=0x7fe784009f10 comp rx=0 tx=0).stop 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 msgr2=0x7fe7941a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ada0 tx=0x7fe78c00c330 comp rx=0 tx=0).stop 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 shutdown_connections 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe794102f80 0x7fe7941a01f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe77c0a3c40 0x7fe77c0a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe794101cc0 0x7fe794198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 --2- 172.21.15.102:0/2080515634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe794102660 0x7fe7941991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 >> 172.21.15.102:0/2080515634 conn(0x7fe7940fc710 msgr2=0x7fe7940fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:13.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 shutdown_connections 2024-04-25T15:01:13.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-25T15:01:13.277+0000 7fe798abd700 1 -- 172.21.15.102:0/2080515634 wait complete. 2024-04-25T15:01:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:13 smithi102 ceph-mon[33005]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:13 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:13.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:13 smithi019 ceph-mon[28090]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:13.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:13.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:13 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:13.937 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T15:01:13.945 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T15:01:13.991 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T15:01:14.037 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T15:01:14.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:13 smithi083 ceph-mon[32919]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:14.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:14.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:13 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:14.083 DEBUG:teuthology.orchestra.run.smithi083:mds.b> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T15:01:14.085 DEBUG:teuthology.orchestra.run.smithi083:mds.e> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T15:01:14.129 DEBUG:teuthology.orchestra.run.smithi083:mds.h> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T15:01:14.171 DEBUG:teuthology.orchestra.run.smithi083:mds.k> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T15:01:14.214 DEBUG:teuthology.orchestra.run.smithi102:mds.c> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T15:01:14.217 DEBUG:teuthology.orchestra.run.smithi102:mds.f> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T15:01:14.260 DEBUG:teuthology.orchestra.run.smithi102:mds.i> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T15:01:14.302 DEBUG:teuthology.orchestra.run.smithi102:mds.l> sudo journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T15:01:14.346 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T15:01:14.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 -- 172.21.15.19:0/4022212966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 msgr2=0x7fd3dc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 --2- 172.21.15.19:0/4022212966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc1043f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009860 tx=0x7fd3cc02a340 comp rx=0 tx=0).stop 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 -- 172.21.15.19:0/4022212966 shutdown_connections 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 --2- 172.21.15.19:0/4022212966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc105b80 0x7fd3dc10c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 --2- 172.21.15.19:0/4022212966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 --2- 172.21.15.19:0/4022212966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc1043f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 -- 172.21.15.19:0/4022212966 >> 172.21.15.19:0/4022212966 conn(0x7fd3dc0ff560 msgr2=0x7fd3dc101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 -- 172.21.15.19:0/4022212966 shutdown_connections 2024-04-25T15:01:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.578+0000 7fd3db200640 1 -- 172.21.15.19:0/4022212966 wait complete. 2024-04-25T15:01:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.579+0000 7fd3db200640 1 Processor -- start 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 -- start start 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10fad0 con 0x7fd3dc1051c0 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10f950 con 0x7fd3dc103fd0 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3db200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc10fc50 con 0x7fd3dc105b80 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3da800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48342/0 (socket says 172.21.15.19:48342) 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:14.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3da800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:35606/0 (socket says 172.21.15.19:35606) 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.580+0000 7fd3d3e00640 1 -- 172.21.15.19:0/1717582515 learned_addr learned my addr 172.21.15.19:0/1717582515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 -- 172.21.15.19:0/1717582515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 msgr2=0x7fd3dc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 -- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 msgr2=0x7fd3dc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 -- 172.21.15.19:0/1717582515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc1a1e50 con 0x7fd3dc105b80 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3d9e00640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3da800640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ef00 tx=0x7fd3d400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004b20 con 0x7fd3dc105b80 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d4003e20 con 0x7fd3dc105b80 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc1a2140 con 0x7fd3dc105b80 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.581+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400b570 con 0x7fd3dc105b80 2024-04-25T15:01:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.582+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a2640 con 0x7fd3dc105b80 2024-04-25T15:01:14.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.583+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc10b410 con 0x7fd3dc105b80 2024-04-25T15:01:14.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.585+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd3d401d020 con 0x7fd3dc105b80 2024-04-25T15:01:14.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.586+0000 7fd3d2a00640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:14.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.586+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd3d40cfe50 con 0x7fd3dc105b80 2024-04-25T15:01:14.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.586+0000 7fd3d9e00640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:14.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.587+0000 7fd3d9e00640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009830 tx=0x7fd3cc030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:14.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.590+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3d4004cc0 con 0x7fd3dc105b80 2024-04-25T15:01:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.799+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3dc062b50 con 0x7fd3dc105b80 2024-04-25T15:01:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.801+0000 7fd3d2a00640 1 -- 172.21.15.19:0/1717582515 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18385 (secure 0 0 0) 0x7fd3d409cc30 con 0x7fd3dc105b80 2024-04-25T15:01:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:14.802 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":84,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:01:03.497686+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 msgr2=0x7fd3bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009830 tx=0x7fd3cc030450 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 msgr2=0x7fd3dc1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ef00 tx=0x7fd3d400c740 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 shutdown_connections 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3dc105b80 0x7fd3dc1a17e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3dc1051c0 0x7fd3dc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.803+0000 7fd3db200640 1 --2- 172.21.15.19:0/1717582515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc103fd0 0x7fd3dc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.804+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 >> 172.21.15.19:0/1717582515 conn(0x7fd3dc0ff560 msgr2=0x7fd3dc1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.804+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 shutdown_connections 2024-04-25T15:01:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:14.804+0000 7fd3db200640 1 -- 172.21.15.19:0/1717582515 wait complete. 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:14 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:14.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:14 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:14.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:14.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 -- 172.21.15.19:0/566715214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 msgr2=0x7fe9380743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 --2- 172.21.15.19:0/566715214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe9380743e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe92802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 -- 172.21.15.19:0/566715214 shutdown_connections 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 --2- 172.21.15.19:0/566715214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe938109e60 0x7fe93810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 --2- 172.21.15.19:0/566715214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9380749b0 0x7fe938074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 --2- 172.21.15.19:0/566715214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe9380743e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 -- 172.21.15.19:0/566715214 >> 172.21.15.19:0/566715214 conn(0x7fe9380fd530 msgr2=0x7fe9380ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 -- 172.21.15.19:0/566715214 shutdown_connections 2024-04-25T15:01:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.039+0000 7fe93e400640 1 -- 172.21.15.19:0/566715214 wait complete. 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:15.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:14 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.040+0000 7fe93e400640 1 Processor -- start 2024-04-25T15:01:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 -- start start 2024-04-25T15:01:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9380749b0 0x7fe93819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe938109e60 0x7fe9381a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810fd20 con 0x7fe9380749b0 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810fba0 con 0x7fe938109e60 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe93e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810fea0 con 0x7fe938075f90 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:35626/0 (socket says 172.21.15.19:35626) 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe936a00640 1 -- 172.21.15.19:0/95408444 learned_addr learned my addr 172.21.15.19:0/95408444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.041+0000 7fe937400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9380749b0 0x7fe93819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe937e00640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe938109e60 0x7fe9381a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 -- 172.21.15.19:0/95408444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe938109e60 msgr2=0x7fe9381a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe938109e60 0x7fe9381a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 -- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9380749b0 msgr2=0x7fe93819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9380749b0 0x7fe93819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 -- 172.21.15.19:0/95408444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9381a2040 con 0x7fe938075f90 2024-04-25T15:01:15.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe936a00640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe92000e9f0 tx=0x7fe92000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.042+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000cd80 con 0x7fe938075f90 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.043+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9381a2330 con 0x7fe938075f90 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.043+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9200054d0 con 0x7fe938075f90 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.043+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920010690 con 0x7fe938075f90 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.043+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9381a2880 con 0x7fe938075f90 2024-04-25T15:01:15.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.044+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938108b90 con 0x7fe938075f90 2024-04-25T15:01:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.047+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe920004e90 con 0x7fe938075f90 2024-04-25T15:01:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.048+0000 7fe92fe00640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 0x7fe91c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.048+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe9200d03f0 con 0x7fe938075f90 2024-04-25T15:01:15.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.048+0000 7fe937400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 0x7fe91c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.049+0000 7fe937400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 0x7fe91c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe928031500 tx=0x7fe928005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.052+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe92009d1d0 con 0x7fe938075f90 2024-04-25T15:01:15.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.259+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe938062b50 con 0x7fe938075f90 2024-04-25T15:01:15.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.260+0000 7fe92fe00640 1 -- 172.21.15.19:0/95408444 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fe92009cad0 con 0x7fe938075f90 2024-04-25T15:01:15.262 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 msgr2=0x7fe91c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 0x7fe91c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe928031500 tx=0x7fe928005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 msgr2=0x7fe93819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe92000e9f0 tx=0x7fe92000edf0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 shutdown_connections 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe91c0a3bb0 0x7fe91c0a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe938109e60 0x7fe9381a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe938075f90 0x7fe93819a960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 --2- 172.21.15.19:0/95408444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9380749b0 0x7fe93819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.263+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 >> 172.21.15.19:0/95408444 conn(0x7fe9380fd530 msgr2=0x7fe938105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.264+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 shutdown_connections 2024-04-25T15:01:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.264+0000 7fe93e400640 1 -- 172.21.15.19:0/95408444 wait complete. 2024-04-25T15:01:15.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:15.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 -- 172.21.15.19:0/4230059858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c101b40 msgr2=0x7f235c101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 --2- 172.21.15.19:0/4230059858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c101b40 0x7f235c101fc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2354009900 tx=0x7f235402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 -- 172.21.15.19:0/4230059858 shutdown_connections 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 --2- 172.21.15.19:0/4230059858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c101b40 0x7f235c101fc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 --2- 172.21.15.19:0/4230059858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c1011a0 0x7f235c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 --2- 172.21.15.19:0/4230059858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c100840 0x7f235c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 -- 172.21.15.19:0/4230059858 >> 172.21.15.19:0/4230059858 conn(0x7f235c0fc440 msgr2=0x7f235c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 -- 172.21.15.19:0/4230059858 shutdown_connections 2024-04-25T15:01:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.500+0000 7f2363200640 1 -- 172.21.15.19:0/4230059858 wait complete. 2024-04-25T15:01:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.501+0000 7f2363200640 1 Processor -- start 2024-04-25T15:01:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.501+0000 7f2363200640 1 -- start start 2024-04-25T15:01:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c1011a0 0x7f235c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c101b40 0x7f235c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10fd20 con 0x7f235c1011a0 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10fba0 con 0x7f235c100840 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2363200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10fea0 con 0x7f235c101b40 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c1011a0 0x7f235c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f2361200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c101b40 0x7f235c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:51454/0 (socket says 172.21.15.19:51454) 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 -- 172.21.15.19:0/4091819981 learned_addr learned my addr 172.21.15.19:0/4091819981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 -- 172.21.15.19:0/4091819981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c101b40 msgr2=0x7f235c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c101b40 0x7f235c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 -- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c1011a0 msgr2=0x7f235c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c1011a0 0x7f235c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.502+0000 7f235be00640 1 -- 172.21.15.19:0/4091819981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1a1f70 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f235be00640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f234c00eb90 tx=0x7f234c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c004d20 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234c00ee90 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1a2260 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c010670 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.503+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1a27b0 con 0x7f235c100840 2024-04-25T15:01:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.504+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c10b590 con 0x7f235c100840 2024-04-25T15:01:15.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.506+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f234c010870 con 0x7f235c100840 2024-04-25T15:01:15.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.508+0000 7f2358c00640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 0x7f23400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:15.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.508+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f234c0d0760 con 0x7f235c100840 2024-04-25T15:01:15.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.508+0000 7f235b400640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 0x7f23400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:15.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.509+0000 7f235b400640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 0x7f23400a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f235c19b860 tx=0x7f2344005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:15.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.510+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f234c053ff0 con 0x7f235c100840 2024-04-25T15:01:15.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.757+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f235c02d5c0 con 0x7f235c100840 2024-04-25T15:01:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.758+0000 7f2358c00640 1 -- 172.21.15.19:0/4091819981 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f234c09d540 con 0x7f235c100840 2024-04-25T15:01:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:15.759 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T15:01:15.759 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-04-25T15:01:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 msgr2=0x7f23400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 0x7f23400a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f235c19b860 tx=0x7f2344005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 msgr2=0x7f235c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f234c00eb90 tx=0x7f234c00c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 shutdown_connections 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c101b40 0x7f235c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f235c1011a0 0x7f235c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f23400a3c30 0x7f23400a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 --2- 172.21.15.19:0/4091819981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f235c100840 0x7f235c19a350 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 >> 172.21.15.19:0/4091819981 conn(0x7f235c0fc440 msgr2=0x7f235c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 shutdown_connections 2024-04-25T15:01:15.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:15.761+0000 7f2363200640 1 -- 172.21.15.19:0/4091819981 wait complete. 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: Deploying daemon mds.l on smithi102 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1717582515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/95408444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: Deploying daemon mds.l on smithi102 2024-04-25T15:01:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1717582515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/95408444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:16.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: Deploying daemon mds.l on smithi102 2024-04-25T15:01:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1717582515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/95408444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4091819981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:16.807 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:16 smithi102 systemd[1]: Starting Ceph mds.l for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4091819981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:17.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4091819981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:17.089 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:16 smithi102 podman[63953]: 2024-04-25 15:01:16.720019938 +0000 UTC m=+0.017969238 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:17.089 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:16 smithi102 podman[63953]: 2024-04-25 15:01:16.844031287 +0000 UTC m=+0.141980559 container create 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T15:01:17.089 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 podman[63953]: 2024-04-25 15:01:17.06330224 +0000 UTC m=+0.361251512 container init 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T15:01:17.089 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 podman[63953]: 2024-04-25 15:01:17.0661088 +0000 UTC m=+0.364058073 container start 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T15:01:17.557 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mds[63972]: -- 172.21.15.102:0/1043120221 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd9870da40 con 0x55fd986e0c00 2024-04-25T15:01:17.557 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l[63968]: starting mds.l at 2024-04-25T15:01:17.557 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 bash[63953]: 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 2024-04-25T15:01:17.557 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:01:17 smithi102 systemd[1]: Started Ceph mds.l for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:17.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.102:6832/1625313784,v1:172.21.15.102:6833/1625313784] up:boot 2024-04-25T15:01:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: fsmap 1 up:standby 2024-04-25T15:01:17.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:17.862 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:17 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.102:6832/1625313784,v1:172.21.15.102:6833/1625313784] up:boot 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: fsmap 1 up:standby 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:17.947 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:17.948 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:17 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 -- 172.21.15.19:0/1689844628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c8103fd0 msgr2=0x7f68c81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 --2- 172.21.15.19:0/1689844628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c8103fd0 0x7f68c81043f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009900 tx=0x7f68bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 -- 172.21.15.19:0/1689844628 shutdown_connections 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 --2- 172.21.15.19:0/1689844628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 --2- 172.21.15.19:0/1689844628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c81051c0 0x7f68c8105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 --2- 172.21.15.19:0/1689844628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c8103fd0 0x7f68c81043f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 -- 172.21.15.19:0/1689844628 >> 172.21.15.19:0/1689844628 conn(0x7f68c80ff560 msgr2=0x7f68c8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 -- 172.21.15.19:0/1689844628 shutdown_connections 2024-04-25T15:01:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.002+0000 7f68ce200640 1 -- 172.21.15.19:0/1689844628 wait complete. 2024-04-25T15:01:18.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.003+0000 7f68ce200640 1 Processor -- start 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 -- start start 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c8103fd0 0x7f68c819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 0x7f68c819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810fb60 con 0x7f68c8105b80 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810f9e0 con 0x7f68c81051c0 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68ce200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810fce0 con 0x7f68c8103fd0 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68cd800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68cd800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c81a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48412/0 (socket says 172.21.15.19:48412) 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 0x7f68c819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68cd800640 1 -- 172.21.15.19:0/2288904326 learned_addr learned my addr 172.21.15.19:0/2288904326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.004+0000 7f68cce00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c8103fd0 0x7f68c819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 -- 172.21.15.19:0/2288904326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c8103fd0 msgr2=0x7f68c819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c8103fd0 0x7f68c819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 -- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 msgr2=0x7f68c81a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c81a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 -- 172.21.15.19:0/2288904326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81a1e40 con 0x7f68c81051c0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c7e00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 0x7f68c819a760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e9f0 tx=0x7f68b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400ccc0 con 0x7f68c81051c0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b4005020 con 0x7f68c81051c0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81a2130 con 0x7f68c81051c0 2024-04-25T15:01:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.005+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4004a00 con 0x7f68c81051c0 2024-04-25T15:01:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.006+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81a2630 con 0x7f68c81051c0 2024-04-25T15:01:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.007+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c810b310 con 0x7f68c81051c0 2024-04-25T15:01:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.010+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f68b4004ba0 con 0x7f68c81051c0 2024-04-25T15:01:18.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.011+0000 7f68c5600640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 0x7f68b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:18.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.011+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f68b40d00f0 con 0x7f68c81051c0 2024-04-25T15:01:18.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.011+0000 7f68cce00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 0x7f68b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:18.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.012+0000 7f68cce00640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 0x7f68b00a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68bc005ab0 tx=0x7f68bc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:18.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.012+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68b409ced0 con 0x7f68c81051c0 2024-04-25T15:01:18.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:18.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.102:6832/1625313784,v1:172.21.15.102:6833/1625313784] up:boot 2024-04-25T15:01:18.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: fsmap 1 up:standby 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:18.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:17 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:18.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.264+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68c802d810 con 0x7f68c81051c0 2024-04-25T15:01:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.265+0000 7f68c5600640 1 -- 172.21.15.19:0/2288904326 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f68b409c7d0 con 0x7f68c81051c0 2024-04-25T15:01:18.266 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:18.266 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T15:01:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-04-25T15:01:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.267+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 msgr2=0x7f68b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.267+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 0x7f68b00a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68bc005ab0 tx=0x7f68bc005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.267+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 msgr2=0x7f68c819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.267+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 0x7f68c819a760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e9f0 tx=0x7f68b400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 shutdown_connections 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68c8105b80 0x7f68c81a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f68b00a3bb0 0x7f68b00a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c81051c0 0x7f68c819a760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 --2- 172.21.15.19:0/2288904326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68c8103fd0 0x7f68c819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:18.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 >> 172.21.15.19:0/2288904326 conn(0x7f68c80ff560 msgr2=0x7f68c81084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:18.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 shutdown_connections 2024-04-25T15:01:18.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:18.268+0000 7f68ce200640 1 -- 172.21.15.19:0/2288904326 wait complete. 2024-04-25T15:01:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:18 smithi019 ceph-mon[28090]: Deploying daemon mds.k on smithi083 2024-04-25T15:01:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2288904326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:18 smithi083 ceph-mon[32919]: Deploying daemon mds.k on smithi083 2024-04-25T15:01:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2288904326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:18 smithi102 ceph-mon[33005]: Deploying daemon mds.k on smithi083 2024-04-25T15:01:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2288904326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:19.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:19 smithi083 ceph-mon[32919]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:19.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:19 smithi019 ceph-mon[28090]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:20.021 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:01:19 smithi083 podman[62202]: 2024-04-25 15:01:19.997682927 +0000 UTC m=+0.359071049 container start 1b6a5c83e209104a589f0f0b455a2f1229f09b03c048a20e1d3ab7aa9995ae33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:01:20.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:19 smithi102 ceph-mon[33005]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:20.282 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:01:20 smithi083 ceph-mds[62221]: -- 172.21.15.83:0/3896766089 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56219361fa40 con 0x5621935f2c00 2024-04-25T15:01:20.283 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:01:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k[62217]: starting mds.k at 2024-04-25T15:01:20.283 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:01:20 smithi083 bash[62202]: 1b6a5c83e209104a589f0f0b455a2f1229f09b03c048a20e1d3ab7aa9995ae33 2024-04-25T15:01:20.283 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:01:20 smithi083 systemd[1]: Started Ceph mds.k for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:20.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 -- 172.21.15.19:0/2776076513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 msgr2=0x7f3900104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 --2- 172.21.15.19:0/2776076513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900104e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 -- 172.21.15.19:0/2776076513 shutdown_connections 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 --2- 172.21.15.19:0/2776076513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 0x7f390010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 --2- 172.21.15.19:0/2776076513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 0x7f3900107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 --2- 172.21.15.19:0/2776076513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900104e40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 -- 172.21.15.19:0/2776076513 >> 172.21.15.19:0/2776076513 conn(0x7f39000fc440 msgr2=0x7f39000fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 -- 172.21.15.19:0/2776076513 shutdown_connections 2024-04-25T15:01:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.567+0000 7f3906200640 1 -- 172.21.15.19:0/2776076513 wait complete. 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.568+0000 7f3906200640 1 Processor -- start 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.568+0000 7f3906200640 1 -- start start 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 0x7f39001970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 0x7f39001936d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f390010db10 con 0x7f3900105380 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f390010d990 con 0x7f3900102a50 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f3906200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f390010dc90 con 0x7f3900107cb0 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:51504/0 (socket says 172.21.15.19:51504) 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 -- 172.21.15.19:0/3194115921 learned_addr learned my addr 172.21.15.19:0/3194115921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38fea00640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 0x7f39001970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ffe00640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 0x7f39001936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 -- 172.21.15.19:0/3194115921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 msgr2=0x7f39001936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 0x7f39001936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 -- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 msgr2=0x7f39001970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 0x7f39001970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.569+0000 7f38ff400640 1 -- 172.21.15.19:0/3194115921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3900193f90 con 0x7f3900102a50 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f38ff400640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005940 tx=0x7f38f00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f003f030 con 0x7f3900102a50 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39001a68b0 con 0x7f3900102a50 2024-04-25T15:01:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f0003cb0 con 0x7f3900102a50 2024-04-25T15:01:20.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0035070 con 0x7f3900102a50 2024-04-25T15:01:20.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.570+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001a6c60 con 0x7f3900102a50 2024-04-25T15:01:20.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.572+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3900109800 con 0x7f3900102a50 2024-04-25T15:01:20.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.574+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38f0042030 con 0x7f3900102a50 2024-04-25T15:01:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.575+0000 7f38f7e00640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 0x7f38e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.575+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f38f00f0300 con 0x7f3900102a50 2024-04-25T15:01:20.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.575+0000 7f38fea00640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 0x7f38e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.576+0000 7f38fea00640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 0x7f38e40a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3900194ae0 tx=0x7f38ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:20.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.583+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f00bca90 con 0x7f3900102a50 2024-04-25T15:01:20.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.847+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390002d5c0 con 0x7f3900102a50 2024-04-25T15:01:20.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.848+0000 7f38f7e00640 1 -- 172.21.15.19:0/3194115921 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f38f00c0d90 con 0x7f3900102a50 2024-04-25T15:01:20.850 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:20.850 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T15:01:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 msgr2=0x7f38e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 0x7f38e40a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3900194ae0 tx=0x7f38ec005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 msgr2=0x7f3900196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005940 tx=0x7f38f00042f0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 shutdown_connections 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3900107cb0 0x7f39001936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3900105380 0x7f39001970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f38e40a3c30 0x7f38e40a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 --2- 172.21.15.19:0/3194115921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3900102a50 0x7f3900196b60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 >> 172.21.15.19:0/3194115921 conn(0x7f39000fc440 msgr2=0x7f3900102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 shutdown_connections 2024-04-25T15:01:20.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:20.851+0000 7f3906200640 1 -- 172.21.15.19:0/3194115921 wait complete. 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:21.454 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: Deploying daemon mds.j on smithi019 2024-04-25T15:01:21.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6832/2427216282,v1:172.21.15.83:6833/2427216282] up:boot 2024-04-25T15:01:21.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: fsmap 2 up:standby 2024-04-25T15:01:21.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:01:21.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:21 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3194115921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:21.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: Deploying daemon mds.j on smithi019 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6832/2427216282,v1:172.21.15.83:6833/2427216282] up:boot 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: fsmap 2 up:standby 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:01:21.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:21 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3194115921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: Deploying daemon mds.j on smithi019 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6832/2427216282,v1:172.21.15.83:6833/2427216282] up:boot 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: fsmap 2 up:standby 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:01:21.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:21 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3194115921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:22.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.182+0000 7fd82ba00640 1 -- 172.21.15.19:0/2276654667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 msgr2=0x7fd8241045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.182+0000 7fd82ba00640 1 --2- 172.21.15.19:0/2276654667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd8241045d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd818009900 tx=0x7fd81802a410 comp rx=0 tx=0).stop 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 -- 172.21.15.19:0/2276654667 shutdown_connections 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 --2- 172.21.15.19:0/2276654667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd824105d60 0x7fd82410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 --2- 172.21.15.19:0/2276654667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8241053a0 0x7fd824105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 --2- 172.21.15.19:0/2276654667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd8241045d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 -- 172.21.15.19:0/2276654667 >> 172.21.15.19:0/2276654667 conn(0x7fd8240ff740 msgr2=0x7fd824101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 -- 172.21.15.19:0/2276654667 shutdown_connections 2024-04-25T15:01:23.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.183+0000 7fd82ba00640 1 -- 172.21.15.19:0/2276654667 wait complete. 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 Processor -- start 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 -- start start 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8241053a0 0x7fd82419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd824105d60 0x7fd8241a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410fd40 con 0x7fd8241041b0 2024-04-25T15:01:23.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410fbc0 con 0x7fd824105d60 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd82ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410fec0 con 0x7fd8241053a0 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd829000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd829000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48454/0 (socket says 172.21.15.19:48454) 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.184+0000 7fd829000640 1 -- 172.21.15.19:0/1876261913 learned_addr learned my addr 172.21.15.19:0/1876261913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd823e00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8241053a0 0x7fd82419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829a00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd824105d60 0x7fd8241a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 -- 172.21.15.19:0/1876261913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8241053a0 msgr2=0x7fd82419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8241053a0 0x7fd82419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 -- 172.21.15.19:0/1876261913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd824105d60 msgr2=0x7fd8241a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd824105d60 0x7fd8241a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 -- 172.21.15.19:0/1876261913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8241a2040 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd829000640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd818031500 tx=0x7fd818003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.185+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81803f070 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.186+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8241a22d0 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.186+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd818031c50 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.186+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8241a27c0 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.186+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818049060 con 0x7fd8241041b0 2024-04-25T15:01:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.187+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82410b5a0 con 0x7fd8241041b0 2024-04-25T15:01:23.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.189+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd818042070 con 0x7fd8241041b0 2024-04-25T15:01:23.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.191+0000 7fd821600640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 0x7fd80c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.191+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd8180f0ec0 con 0x7fd8241041b0 2024-04-25T15:01:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.191+0000 7fd823e00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 0x7fd80c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.192+0000 7fd823e00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 0x7fd80c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd82419b930 tx=0x7fd810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.198+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8180bd650 con 0x7fd8241041b0 2024-04-25T15:01:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:23 smithi019 ceph-mon[28090]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:23.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.452+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd82402d5c0 con 0x7fd8241041b0 2024-04-25T15:01:23.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.452+0000 7fd821600640 1 -- 172.21.15.19:0/1876261913 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd8180c1950 con 0x7fd8241041b0 2024-04-25T15:01:23.454 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:23.454 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T15:01:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T15:01:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 msgr2=0x7fd80c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 0x7fd80c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd82419b930 tx=0x7fd810005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 msgr2=0x7fd82419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd818031500 tx=0x7fd818003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 shutdown_connections 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd80c0a3bb0 0x7fd80c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd824105d60 0x7fd8241a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8241053a0 0x7fd82419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 --2- 172.21.15.19:0/1876261913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8241041b0 0x7fd82419a420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 >> 172.21.15.19:0/1876261913 conn(0x7fd8240ff740 msgr2=0x7fd824108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 shutdown_connections 2024-04-25T15:01:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:23.455+0000 7fd82ba00640 1 -- 172.21.15.19:0/1876261913 wait complete. 2024-04-25T15:01:23.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:23 smithi083 ceph-mon[32919]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:23 smithi102 ceph-mon[33005]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:24.106 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:23 smithi019 podman[68106]: 2024-04-25 15:01:23.800171786 +0000 UTC m=+0.140109288 container create edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:01:24.106 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 podman[68106]: 2024-04-25 15:01:24.104215576 +0000 UTC m=+0.444153088 container init edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T15:01:24.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:24 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1876261913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.372 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 podman[68106]: 2024-04-25 15:01:24.115589239 +0000 UTC m=+0.455526728 container start edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:01:24.372 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 ceph-mds[68125]: -- 172.21.15.19:0/1312882563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cd721fa40 con 0x556cd71f3000 2024-04-25T15:01:24.372 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j[68121]: starting mds.j at 2024-04-25T15:01:24.372 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 bash[68106]: edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 2024-04-25T15:01:24.372 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:01:24 smithi019 systemd[1]: Started Ceph mds.j for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:24.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:24 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1876261913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:24 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1876261913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:25.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:25.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:25.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: Deploying daemon mds.d on smithi019 2024-04-25T15:01:25.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.19:6834/1454655799,v1:172.21.15.19:6835/1454655799] up:boot 2024-04-25T15:01:25.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: fsmap 3 up:standby 2024-04-25T15:01:25.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:25 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:01:25.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 -- 172.21.15.19:0/3334743976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 msgr2=0x7f1c28108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 --2- 172.21.15.19:0/3334743976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c28108810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009900 tx=0x7f1c2002a410 comp rx=0 tx=0).stop 2024-04-25T15:01:25.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 -- 172.21.15.19:0/3334743976 shutdown_connections 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 --2- 172.21.15.19:0/3334743976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c28108810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 --2- 172.21.15.19:0/3334743976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 --2- 172.21.15.19:0/3334743976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c281011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 -- 172.21.15.19:0/3334743976 >> 172.21.15.19:0/3334743976 conn(0x7f1c280fb580 msgr2=0x7f1c280fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 -- 172.21.15.19:0/3334743976 shutdown_connections 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.693+0000 7f1c27200640 1 -- 172.21.15.19:0/3334743976 wait complete. 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.694+0000 7f1c27200640 1 Processor -- start 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.694+0000 7f1c27200640 1 -- start start 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c2819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d930 con 0x7f1c28106420 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d7b0 con 0x7f1c28068400 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c27200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810dab0 con 0x7f1c28101720 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c25e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c1d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c25e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:38706/0 (socket says 172.21.15.19:38706) 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c25e00640 1 -- 172.21.15.19:0/716703337 learned_addr learned my addr 172.21.15.19:0/716703337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.695+0000 7f1c26800640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c2819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 -- 172.21.15.19:0/716703337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 msgr2=0x7f1c28198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 -- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 msgr2=0x7f1c2819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c2819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 -- 172.21.15.19:0/716703337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2819fbe0 con 0x7f1c28068400 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c26800640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c2819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c1d400640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c25e00640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009d20 tx=0x7f1c1800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18010630 con 0x7f1c28068400 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2819fed0 con 0x7f1c28068400 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c18010c80 con 0x7f1c28068400 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18018d50 con 0x7f1c28068400 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.696+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a0390 con 0x7f1c28068400 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.698+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c28105230 con 0x7f1c28068400 2024-04-25T15:01:25.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.700+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c1801f020 con 0x7f1c28068400 2024-04-25T15:01:25.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.701+0000 7f1c1f400640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.702+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1c180d07c0 con 0x7f1c28068400 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.702+0000 7f1c1d400640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.702+0000 7f1c1d400640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c281994d0 tx=0x7f1c10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.704+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c1809d5a0 con 0x7f1c28068400 2024-04-25T15:01:25.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: Deploying daemon mds.d on smithi019 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.19:6834/1454655799,v1:172.21.15.19:6835/1454655799] up:boot 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: fsmap 3 up:standby 2024-04-25T15:01:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:25 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:25.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: Deploying daemon mds.d on smithi019 2024-04-25T15:01:25.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.19:6834/1454655799,v1:172.21.15.19:6835/1454655799] up:boot 2024-04-25T15:01:25.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: fsmap 3 up:standby 2024-04-25T15:01:25.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:25 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:01:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.959+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c2802d5c0 con 0x7f1c28068400 2024-04-25T15:01:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.960+0000 7f1c1f400640 1 -- 172.21.15.19:0/716703337 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f1c1809cea0 con 0x7f1c28068400 2024-04-25T15:01:25.961 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:25.961 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T15:01:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-04-25T15:01:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 msgr2=0x7f1c0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c281994d0 tx=0x7f1c10005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 msgr2=0x7f1c28197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009d20 tx=0x7f1c1800c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 shutdown_connections 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28106420 0x7f1c2819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c28101720 0x7f1c28198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 --2- 172.21.15.19:0/716703337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c28068400 0x7f1c28197fc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 >> 172.21.15.19:0/716703337 conn(0x7f1c280fb580 msgr2=0x7f1c28102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 shutdown_connections 2024-04-25T15:01:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:25.962+0000 7f1c27200640 1 -- 172.21.15.19:0/716703337 wait complete. 2024-04-25T15:01:26.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:26 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/716703337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:26.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:26 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/716703337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:26 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/716703337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:27 smithi083 ceph-mon[32919]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:27.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:27 smithi102 ceph-mon[33005]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:27.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:27 smithi019 ceph-mon[28090]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:27.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 -- 172.21.15.19:0/1402328497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 msgr2=0x7f6e60105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 --2- 172.21.15.19:0/1402328497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e60105820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009900 tx=0x7f6e4c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 -- 172.21.15.19:0/1402328497 shutdown_connections 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 --2- 172.21.15.19:0/1402328497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 0x7f6e6010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 --2- 172.21.15.19:0/1402328497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e60105820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 --2- 172.21.15.19:0/1402328497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e601045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 -- 172.21.15.19:0/1402328497 >> 172.21.15.19:0/1402328497 conn(0x7f6e600ff740 msgr2=0x7f6e60101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 -- 172.21.15.19:0/1402328497 shutdown_connections 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.190+0000 7f6e64a00640 1 -- 172.21.15.19:0/1402328497 wait complete. 2024-04-25T15:01:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.191+0000 7f6e64a00640 1 Processor -- start 2024-04-25T15:01:28.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.192+0000 7f6e64a00640 1 -- start start 2024-04-25T15:01:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e6019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 0x7f6e601a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010fd40 con 0x7f6e601053a0 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010fbc0 con 0x7f6e601041b0 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e64a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010fec0 con 0x7f6e60105d60 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56918/0 (socket says 172.21.15.19:56918) 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e5d400640 1 -- 172.21.15.19:0/617940664 learned_addr learned my addr 172.21.15.19:0/617940664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e5de00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e6019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.193+0000 7f6e5e800640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 0x7f6e601a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 -- 172.21.15.19:0/617940664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 msgr2=0x7f6e601a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 0x7f6e601a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 -- 172.21.15.19:0/617940664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 msgr2=0x7f6e6019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e6019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 -- 172.21.15.19:0/617940664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601a1fe0 con 0x7f6e601053a0 2024-04-25T15:01:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5de00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e6019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e5d400640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005ab0 tx=0x7f6e4c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c03f070 con 0x7f6e601053a0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e4c031c50 con 0x7f6e601053a0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601a2270 con 0x7f6e601053a0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c035030 con 0x7f6e601053a0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.194+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601a27b0 con 0x7f6e601053a0 2024-04-25T15:01:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.196+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6010b4d0 con 0x7f6e601053a0 2024-04-25T15:01:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.199+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e4c042070 con 0x7f6e601053a0 2024-04-25T15:01:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.201+0000 7f6e56a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 0x7f6e440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.202+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6e4c0f0e00 con 0x7f6e601053a0 2024-04-25T15:01:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.202+0000 7f6e5de00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 0x7f6e440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.203+0000 7f6e5de00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 0x7f6e440a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e60105200 tx=0x7f6e50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.204+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e4c031e00 con 0x7f6e601053a0 2024-04-25T15:01:28.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.460+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e6002d5c0 con 0x7f6e601053a0 2024-04-25T15:01:28.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.461+0000 7f6e56a00640 1 -- 172.21.15.19:0/617940664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f6e4c0bd590 con 0x7f6e601053a0 2024-04-25T15:01:28.463 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:28.463 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T15:01:28.463 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-04-25T15:01:28.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 msgr2=0x7f6e440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:28.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 0x7f6e440a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e60105200 tx=0x7f6e50005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:28.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 msgr2=0x7f6e6019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:28.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005ab0 tx=0x7f6e4c003b90 comp rx=0 tx=0).stop 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 shutdown_connections 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e60105d60 0x7f6e601a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e601053a0 0x7f6e6019a900 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e440a3c30 0x7f6e440a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 --2- 172.21.15.19:0/617940664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e601041b0 0x7f6e6019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.463+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 >> 172.21.15.19:0/617940664 conn(0x7f6e600ff740 msgr2=0x7f6e601086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.464+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 shutdown_connections 2024-04-25T15:01:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:28.464+0000 7f6e64a00640 1 -- 172.21.15.19:0/617940664 wait complete. 2024-04-25T15:01:29.515 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:29.515 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.19:6836/3801236532,v1:172.21.15.19:6837/3801236532] up:boot 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: fsmap 4 up:standby 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/617940664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:29.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:29 smithi083 ceph-mon[32919]: Deploying daemon mds.e on smithi083 2024-04-25T15:01:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.19:6836/3801236532,v1:172.21.15.19:6837/3801236532] up:boot 2024-04-25T15:01:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: fsmap 4 up:standby 2024-04-25T15:01:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:01:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/617940664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:29.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:29 smithi102 ceph-mon[33005]: Deploying daemon mds.e on smithi083 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.19:6836/3801236532,v1:172.21.15.19:6837/3801236532] up:boot 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: fsmap 4 up:standby 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/617940664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:29.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:29.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:29.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:29.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:29 smithi019 ceph-mon[28090]: Deploying daemon mds.e on smithi083 2024-04-25T15:01:30.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:30.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 -- 172.21.15.19:0/1513532226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca81053c0 msgr2=0x7fdca8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 --2- 172.21.15.19:0/1513532226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca81053c0 0x7fdca8105840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009900 tx=0x7fdc9002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 -- 172.21.15.19:0/1513532226 shutdown_connections 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 --2- 172.21.15.19:0/1513532226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca8105d80 0x7fdca810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 --2- 172.21.15.19:0/1513532226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca81053c0 0x7fdca8105840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 --2- 172.21.15.19:0/1513532226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 0x7fdca81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 -- 172.21.15.19:0/1513532226 >> 172.21.15.19:0/1513532226 conn(0x7fdca80ff740 msgr2=0x7fdca8101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 -- 172.21.15.19:0/1513532226 shutdown_connections 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.708+0000 7fdcacc00640 1 -- 172.21.15.19:0/1513532226 wait complete. 2024-04-25T15:01:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.709+0000 7fdcacc00640 1 Processor -- start 2024-04-25T15:01:30.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 -- start start 2024-04-25T15:01:30.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 0x7fdca819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca8105d80 0x7fdca81a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810fd60 con 0x7fdca81041d0 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810fbe0 con 0x7fdca8105d80 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.710+0000 7fdcacc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810fee0 con 0x7fdca81053c0 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:60072/0 (socket says 172.21.15.19:60072) 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 -- 172.21.15.19:0/3700635371 learned_addr learned my addr 172.21.15.19:0/3700635371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5e00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 0x7fdca819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca6800640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca8105d80 0x7fdca81a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 -- 172.21.15.19:0/3700635371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca8105d80 msgr2=0x7fdca81a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca8105d80 0x7fdca81a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 -- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 msgr2=0x7fdca819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 0x7fdca819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 -- 172.21.15.19:0/3700635371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca81a2020 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.711+0000 7fdca5400640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc90005ab0 tx=0x7fdc90003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.712+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9003f070 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.712+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc90031c50 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.712+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca81a22b0 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.712+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90035030 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.712+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca81a27a0 con 0x7fdca81053c0 2024-04-25T15:01:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.713+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca810b570 con 0x7fdca81053c0 2024-04-25T15:01:30.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.715+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc90042070 con 0x7fdca81053c0 2024-04-25T15:01:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.716+0000 7fdc9ea00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 0x7fdc8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.717+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdc900f0dc0 con 0x7fdca81053c0 2024-04-25T15:01:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.717+0000 7fdca5e00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 0x7fdc8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.717+0000 7fdca5e00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 0x7fdc8c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdca8105220 tx=0x7fdc98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:30.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.719+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc900bd550 con 0x7fdca81053c0 2024-04-25T15:01:30.790 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:30 smithi083 systemd[1]: Starting Ceph mds.e for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.967+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca802d810 con 0x7fdca81053c0 2024-04-25T15:01:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.968+0000 7fdc9ea00640 1 -- 172.21.15.19:0/3700635371 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fdc900c1850 con 0x7fdca81053c0 2024-04-25T15:01:30.969 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:30.969 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T15:01:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-04-25T15:01:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 msgr2=0x7fdc8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 0x7fdc8c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdca8105220 tx=0x7fdc98009500 comp rx=0 tx=0).stop 2024-04-25T15:01:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 msgr2=0x7fdca819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc90005ab0 tx=0x7fdc90003b90 comp rx=0 tx=0).stop 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 shutdown_connections 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdc8c0a3bb0 0x7fdc8c0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdca8105d80 0x7fdca81a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdca81053c0 0x7fdca819a940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 --2- 172.21.15.19:0/3700635371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdca81041d0 0x7fdca819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 >> 172.21.15.19:0/3700635371 conn(0x7fdca80ff740 msgr2=0x7fdca8108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 shutdown_connections 2024-04-25T15:01:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:30.971+0000 7fdcacc00640 1 -- 172.21.15.19:0/3700635371 wait complete. 2024-04-25T15:01:31.049 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:30 smithi083 podman[62521]: 2024-04-25 15:01:30.700214916 +0000 UTC m=+0.017862710 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:31.050 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:30 smithi083 podman[62521]: 2024-04-25 15:01:30.804223235 +0000 UTC m=+0.121871024 container create 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:01:31.050 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 podman[62521]: 2024-04-25 15:01:31.023243433 +0000 UTC m=+0.340891222 container init 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:01:31.050 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 podman[62521]: 2024-04-25 15:01:31.025950623 +0000 UTC m=+0.343598415 container start 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T15:01:31.050 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mds[62540]: -- 172.21.15.83:0/2960059670 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557275051a40 con 0x557275024c00 2024-04-25T15:01:31.387 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e[62536]: starting mds.e at 2024-04-25T15:01:31.387 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 bash[62521]: 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 2024-04-25T15:01:31.387 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:01:31 smithi083 systemd[1]: Started Ceph mds.e for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:31.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3700635371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:31.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3700635371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3700635371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:31.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:32.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:32 smithi083 ceph-mon[32919]: Deploying daemon mds.f on smithi102 2024-04-25T15:01:32.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:32 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6834/334283941,v1:172.21.15.83:6835/334283941] up:boot 2024-04-25T15:01:32.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:32 smithi083 ceph-mon[32919]: fsmap 5 up:standby 2024-04-25T15:01:32.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:32 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:01:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:32 smithi102 ceph-mon[33005]: Deploying daemon mds.f on smithi102 2024-04-25T15:01:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:32 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6834/334283941,v1:172.21.15.83:6835/334283941] up:boot 2024-04-25T15:01:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:32 smithi102 ceph-mon[33005]: fsmap 5 up:standby 2024-04-25T15:01:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:32 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:01:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:32 smithi019 ceph-mon[28090]: Deploying daemon mds.f on smithi102 2024-04-25T15:01:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:32 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6834/334283941,v1:172.21.15.83:6835/334283941] up:boot 2024-04-25T15:01:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:32 smithi019 ceph-mon[28090]: fsmap 5 up:standby 2024-04-25T15:01:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:32 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:01:32.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 -- 172.21.15.19:0/88628494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc103190 msgr2=0x7f81dc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 --2- 172.21.15.19:0/88628494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc103190 0x7f81dc103610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009900 tx=0x7f81c802a410 comp rx=0 tx=0).stop 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 -- 172.21.15.19:0/88628494 shutdown_connections 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 --2- 172.21.15.19:0/88628494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103b50 0x7f81dc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 --2- 172.21.15.19:0/88628494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc103190 0x7f81dc103610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 --2- 172.21.15.19:0/88628494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc101fa0 0x7f81dc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 -- 172.21.15.19:0/88628494 >> 172.21.15.19:0/88628494 conn(0x7f81dc0fd530 msgr2=0x7f81dc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.213+0000 7f81e3a00640 1 -- 172.21.15.19:0/88628494 shutdown_connections 2024-04-25T15:01:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.214+0000 7f81e3a00640 1 -- 172.21.15.19:0/88628494 wait complete. 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.214+0000 7f81e3a00640 1 Processor -- start 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 -- start start 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc101fa0 0x7f81dc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 0x7f81dc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10db30 con 0x7f81dc103190 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10d9b0 con 0x7f81dc103b50 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e3a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10dcb0 con 0x7f81dc101fa0 2024-04-25T15:01:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56976/0 (socket says 172.21.15.19:56976) 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81d3e00640 1 -- 172.21.15.19:0/957003956 learned_addr learned my addr 172.21.15.19:0/957003956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e1a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 0x7f81dc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.215+0000 7f81e1000640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc101fa0 0x7f81dc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 -- 172.21.15.19:0/957003956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc101fa0 msgr2=0x7f81dc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc101fa0 0x7f81dc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 -- 172.21.15.19:0/957003956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 msgr2=0x7f81dc074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 0x7f81dc074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 -- 172.21.15.19:0/957003956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc075110 con 0x7f81dc103190 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81e1a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 0x7f81dc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81d3e00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81c8005ab0 tx=0x7f81c8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c803f070 con 0x7f81dc103190 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c8031c50 con 0x7f81dc103190 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8035030 con 0x7f81dc103190 2024-04-25T15:01:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.216+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc1a2280 con 0x7f81dc103190 2024-04-25T15:01:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.217+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a2820 con 0x7f81dc103190 2024-04-25T15:01:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.218+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc109340 con 0x7f81dc103190 2024-04-25T15:01:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.221+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81c8042070 con 0x7f81dc103190 2024-04-25T15:01:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.222+0000 7f81da000640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 0x7f81c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.222+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f81c80f0e70 con 0x7f81dc103190 2024-04-25T15:01:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.222+0000 7f81e1000640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 0x7f81c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.223+0000 7f81e1000640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 0x7f81c00a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81dc102ff0 tx=0x7f81cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:33.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.224+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81c80bdc50 con 0x7f81dc103190 2024-04-25T15:01:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.471+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81dc02d810 con 0x7f81dc103190 2024-04-25T15:01:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.472+0000 7f81da000640 1 -- 172.21.15.19:0/957003956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f81c8031e00 con 0x7f81dc103190 2024-04-25T15:01:33.474 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:33.474 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T15:01:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.475+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 msgr2=0x7f81c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.475+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 0x7f81c00a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81dc102ff0 tx=0x7f81cc009500 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.475+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 msgr2=0x7f81dc074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.475+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81c8005ab0 tx=0x7f81c8003b90 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 shutdown_connections 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f81c00a3c30 0x7f81c00a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81dc103b50 0x7f81dc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81dc103190 0x7f81dc074340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 --2- 172.21.15.19:0/957003956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81dc101fa0 0x7f81dc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 >> 172.21.15.19:0/957003956 conn(0x7f81dc0fd530 msgr2=0x7f81dc106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 shutdown_connections 2024-04-25T15:01:33.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:33.476+0000 7f81e3a00640 1 -- 172.21.15.19:0/957003956 wait complete. 2024-04-25T15:01:33.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:33 smithi102 ceph-mon[33005]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:33.549 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:33 smithi102 systemd[1]: Starting Ceph mds.f for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:33.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:33 smithi083 ceph-mon[32919]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:33.807 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:33 smithi102 podman[64260]: 2024-04-25 15:01:33.764489525 +0000 UTC m=+0.110740854 container create 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:01:33.807 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:33 smithi102 podman[64260]: 2024-04-25 15:01:33.671686815 +0000 UTC m=+0.017938156 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:33.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:33 smithi019 ceph-mon[28090]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:34.307 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:33 smithi102 podman[64260]: 2024-04-25 15:01:33.975572151 +0000 UTC m=+0.321823480 container init 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:01:34.307 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:33 smithi102 podman[64260]: 2024-04-25 15:01:33.978132499 +0000 UTC m=+0.324383837 container start 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T15:01:34.308 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mds[64279]: -- 172.21.15.102:0/1836492926 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b336a5a40 con 0x556b33679000 2024-04-25T15:01:34.308 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f[64275]: starting mds.f at 2024-04-25T15:01:34.308 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:34 smithi102 bash[64260]: 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d 2024-04-25T15:01:34.308 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:01:34 smithi102 systemd[1]: Started Ceph mds.f for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:34.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/957003956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:34 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/957003956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:34.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:34.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:34 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:34.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/957003956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:34 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:35.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 -- 172.21.15.19:0/283995041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 msgr2=0x7f4660105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 --2- 172.21.15.19:0/283995041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f4660105640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 -- 172.21.15.19:0/283995041 shutdown_connections 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 --2- 172.21.15.19:0/283995041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f466010c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 --2- 172.21.15.19:0/283995041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f4660105640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 --2- 172.21.15.19:0/283995041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f46601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 -- 172.21.15.19:0/283995041 >> 172.21.15.19:0/283995041 conn(0x7f46600ff560 msgr2=0x7f4660101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 -- 172.21.15.19:0/283995041 shutdown_connections 2024-04-25T15:01:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.708+0000 7f4665e00640 1 -- 172.21.15.19:0/283995041 wait complete. 2024-04-25T15:01:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.710+0000 7f4665e00640 1 Processor -- start 2024-04-25T15:01:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.710+0000 7f4665e00640 1 -- start start 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f4660079cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f466007a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f466010fb60 con 0x7f46601051c0 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f466010f9e0 con 0x7f4660103fd0 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f466010fce0 con 0x7f4660105b80 2024-04-25T15:01:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f465fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f466007a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f4660079cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:41678/0 (socket says 172.21.15.19:41678) 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665400640 1 -- 172.21.15.19:0/2030201048 learned_addr learned my addr 172.21.15.19:0/2030201048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f465fe00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f466007a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43534/0 (socket says 172.21.15.19:43534) 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 -- 172.21.15.19:0/2030201048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 msgr2=0x7f4660076820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 -- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 msgr2=0x7f466007a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f466007a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4664a00640 1 -- 172.21.15.19:0/2030201048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46600770e0 con 0x7f4660103fd0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.711+0000 7f4665400640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f4664a00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f4660079cb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400ccc0 con 0x7f4660103fd0 2024-04-25T15:01:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46540035f0 con 0x7f4660103fd0 2024-04-25T15:01:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601a6660 con 0x7f4660103fd0 2024-04-25T15:01:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400ce60 con 0x7f4660103fd0 2024-04-25T15:01:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.712+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601a6a10 con 0x7f4660103fd0 2024-04-25T15:01:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.713+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466010b2f0 con 0x7f4660103fd0 2024-04-25T15:01:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.715+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4654004000 con 0x7f4660103fd0 2024-04-25T15:01:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.717+0000 7f465d600640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 0x7f46480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.717+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f46540d0430 con 0x7f4660103fd0 2024-04-25T15:01:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.717+0000 7f465fe00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 0x7f46480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.718+0000 7f465fe00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 0x7f46480a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f464c031500 tx=0x7f464c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.721+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f465409d210 con 0x7f4660103fd0 2024-04-25T15:01:35.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:35 smithi083 ceph-mon[32919]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:35.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:35 smithi083 ceph-mon[32919]: Deploying daemon mds.c on smithi102 2024-04-25T15:01:35.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:35 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.102:6834/3800814259,v1:172.21.15.102:6835/3800814259] up:boot 2024-04-25T15:01:35.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:35 smithi083 ceph-mon[32919]: fsmap 6 up:standby 2024-04-25T15:01:35.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:35 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:01:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:35 smithi102 ceph-mon[33005]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:35 smithi102 ceph-mon[33005]: Deploying daemon mds.c on smithi102 2024-04-25T15:01:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:35 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.102:6834/3800814259,v1:172.21.15.102:6835/3800814259] up:boot 2024-04-25T15:01:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:35 smithi102 ceph-mon[33005]: fsmap 6 up:standby 2024-04-25T15:01:35.799 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:35 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:01:35.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:35 smithi019 ceph-mon[28090]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:35.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:35 smithi019 ceph-mon[28090]: Deploying daemon mds.c on smithi102 2024-04-25T15:01:35.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:35 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.102:6834/3800814259,v1:172.21.15.102:6835/3800814259] up:boot 2024-04-25T15:01:35.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:35 smithi019 ceph-mon[28090]: fsmap 6 up:standby 2024-04-25T15:01:35.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:35 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:01:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.967+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4660062b50 con 0x7f4660103fd0 2024-04-25T15:01:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.968+0000 7f465d600640 1 -- 172.21.15.19:0/2030201048 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f465409cb10 con 0x7f4660103fd0 2024-04-25T15:01:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:35.970 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T15:01:35.970 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-25T15:01:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 msgr2=0x7f46480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 0x7f46480a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f464c031500 tx=0x7f464c005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 msgr2=0x7f4660079cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f4660079cb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).stop 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 shutdown_connections 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4660105b80 0x7f4660076820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601051c0 0x7f466007a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f46480a3bb0 0x7f46480a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 --2- 172.21.15.19:0/2030201048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4660103fd0 0x7f4660079cb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 >> 172.21.15.19:0/2030201048 conn(0x7f46600ff560 msgr2=0x7f46601084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 shutdown_connections 2024-04-25T15:01:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:35.971+0000 7f4665e00640 1 -- 172.21.15.19:0/2030201048 wait complete. 2024-04-25T15:01:36.438 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 systemd[1]: Starting Ceph mds.c for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:36 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2030201048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 podman[64570]: 2024-04-25 15:01:36.384814876 +0000 UTC m=+0.018051916 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 podman[64570]: 2024-04-25 15:01:36.48474899 +0000 UTC m=+0.117986051 container create f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 podman[64570]: 2024-04-25 15:01:36.678746198 +0000 UTC m=+0.311983217 container init f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 podman[64570]: 2024-04-25 15:01:36.681542662 +0000 UTC m=+0.314779689 container start f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:01:36.705 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 ceph-mds[64589]: -- 172.21.15.102:0/2710449692 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1d9dc1a40 con 0x55f1d9d94800 2024-04-25T15:01:36.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:36 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2030201048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:36.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:36 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2030201048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:37.057 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c[64585]: starting mds.c at 2024-04-25T15:01:37.058 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 bash[64570]: f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 2024-04-25T15:01:37.058 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:01:36 smithi102 systemd[1]: Started Ceph mds.c for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:37.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.102:6836/560569157,v1:172.21.15.102:6837/560569157] up:boot 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: fsmap 7 up:standby 2024-04-25T15:01:38.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:37 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 -- 172.21.15.19:0/2679333498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc0105d80 msgr2=0x7fecc010c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 --2- 172.21.15.19:0/2679333498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc0105d80 0x7fecc010c610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009900 tx=0x7fecb802a410 comp rx=0 tx=0).stop 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 -- 172.21.15.19:0/2679333498 shutdown_connections 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 --2- 172.21.15.19:0/2679333498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc0105d80 0x7fecc010c610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 --2- 172.21.15.19:0/2679333498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01053c0 0x7fecc0105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 --2- 172.21.15.19:0/2679333498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc01041d0 0x7fecc01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.214+0000 7fecc6c00640 1 -- 172.21.15.19:0/2679333498 >> 172.21.15.19:0/2679333498 conn(0x7fecc00ff740 msgr2=0x7fecc0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.215+0000 7fecc6c00640 1 -- 172.21.15.19:0/2679333498 shutdown_connections 2024-04-25T15:01:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.215+0000 7fecc6c00640 1 -- 172.21.15.19:0/2679333498 wait complete. 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.215+0000 7fecc6c00640 1 Processor -- start 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 -- start start 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 0x7fecc019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 0x7fecc019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 0x7fecc01a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010fd60 con 0x7fecc01041d0 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010fbe0 con 0x7fecc01053c0 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010fee0 con 0x7fecc0105d80 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecbfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 0x7fecc019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecbf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 0x7fecc019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecbfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 0x7fecc019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43556/0 (socket says 172.21.15.19:43556) 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecbfe00640 1 -- 172.21.15.19:0/3895820 learned_addr learned my addr 172.21.15.19:0/3895820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.216+0000 7fecc4c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 0x7fecc01a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 -- 172.21.15.19:0/3895820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 msgr2=0x7fecc01a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 0x7fecc01a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 -- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 msgr2=0x7fecc019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 0x7fecc019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 -- 172.21.15.19:0/3895820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc01a1fb0 con 0x7fecc01053c0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecc4c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 0x7fecc01a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbf400640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 0x7fecc019a8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9f0 tx=0x7feca800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800ccc0 con 0x7fecc01053c0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca80108e0 con 0x7fecc01053c0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8018880 con 0x7fecc01053c0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc01a22a0 con 0x7fecc01053c0 2024-04-25T15:01:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.217+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc01a27a0 con 0x7fecc01053c0 2024-04-25T15:01:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.218+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc010b560 con 0x7fecc01053c0 2024-04-25T15:01:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.220+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feca8010430 con 0x7fecc01053c0 2024-04-25T15:01:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.222+0000 7fecbcc00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 0x7feca40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.222+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7feca80d03a0 con 0x7fecc01053c0 2024-04-25T15:01:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.222+0000 7fecbfe00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 0x7feca40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.223+0000 7fecbfe00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 0x7feca40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fecb0011440 tx=0x7fecb0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:38.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.226+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feca809d180 con 0x7fecc01053c0 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:38.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:38.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.102:6836/560569157,v1:172.21.15.102:6837/560569157] up:boot 2024-04-25T15:01:38.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: fsmap 7 up:standby 2024-04-25T15:01:38.292 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:37 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:38.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:38.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:38.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.102:6836/560569157,v1:172.21.15.102:6837/560569157] up:boot 2024-04-25T15:01:38.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: fsmap 7 up:standby 2024-04-25T15:01:38.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:37 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:01:38.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.476+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc0062b80 con 0x7fecc01053c0 2024-04-25T15:01:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.477+0000 7fecbcc00640 1 -- 172.21.15.19:0/3895820 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7feca80bd0c0 con 0x7fecc01053c0 2024-04-25T15:01:38.478 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:38.478 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T15:01:38.479 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 msgr2=0x7feca40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 0x7feca40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fecb0011440 tx=0x7fecb0009500 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 msgr2=0x7fecc019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 0x7fecc019a8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9f0 tx=0x7feca800eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 shutdown_connections 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0105d80 0x7fecc01a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7feca40a3bb0 0x7feca40a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc01053c0 0x7fecc019a8d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 --2- 172.21.15.19:0/3895820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecc01041d0 0x7fecc019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.479+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 >> 172.21.15.19:0/3895820 conn(0x7fecc00ff740 msgr2=0x7fecc0108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:38.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.480+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 shutdown_connections 2024-04-25T15:01:38.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:38.480+0000 7fecc6c00640 1 -- 172.21.15.19:0/3895820 wait complete. 2024-04-25T15:01:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:38 smithi083 ceph-mon[32919]: Deploying daemon mds.h on smithi083 2024-04-25T15:01:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:38 smithi083 ceph-mon[32919]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:38 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3895820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:39.042 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:38 smithi083 systemd[1]: Starting Ceph mds.h for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:39.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:38 smithi019 ceph-mon[28090]: Deploying daemon mds.h on smithi083 2024-04-25T15:01:39.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:38 smithi019 ceph-mon[28090]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:39.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:38 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3895820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:39.208 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:38 smithi102 ceph-mon[33005]: Deploying daemon mds.h on smithi083 2024-04-25T15:01:39.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:38 smithi102 ceph-mon[33005]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:39.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:38 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3895820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:39.336 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 podman[62827]: 2024-04-25 15:01:38.971995673 +0000 UTC m=+0.017575717 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:39.336 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 podman[62827]: 2024-04-25 15:01:39.092795294 +0000 UTC m=+0.138375321 container create d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:01:39.336 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 podman[62827]: 2024-04-25 15:01:39.310200118 +0000 UTC m=+0.355780158 container init d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T15:01:39.336 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 podman[62827]: 2024-04-25 15:01:39.31280167 +0000 UTC m=+0.358381702 container start d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:01:39.790 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 ceph-mds[62846]: -- 172.21.15.83:0/1808314023 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644e930fa40 con 0x5644e92e2800 2024-04-25T15:01:39.791 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h[62842]: starting mds.h at 2024-04-25T15:01:39.791 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 bash[62827]: d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 2024-04-25T15:01:39.791 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:01:39 smithi083 systemd[1]: Started Ceph mds.h for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:40.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:40.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.511 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: Deploying daemon mds.i on smithi102 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6836/3676268156,v1:172.21.15.83:6837/3676268156] up:boot 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: fsmap 8 up:standby 2024-04-25T15:01:40.512 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:40 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 -- 172.21.15.19:0/4155253111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d4105b80 msgr2=0x7fc0d410c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 --2- 172.21.15.19:0/4155253111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d4105b80 0x7fc0d410c410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009860 tx=0x7fc0cc02a340 comp rx=0 tx=0).stop 2024-04-25T15:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 -- 172.21.15.19:0/4155253111 shutdown_connections 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 --2- 172.21.15.19:0/4155253111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d4105b80 0x7fc0d410c410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 --2- 172.21.15.19:0/4155253111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d41051c0 0x7fc0d4105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 --2- 172.21.15.19:0/4155253111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 0x7fc0d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 -- 172.21.15.19:0/4155253111 >> 172.21.15.19:0/4155253111 conn(0x7fc0d40ff560 msgr2=0x7fc0d4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 -- 172.21.15.19:0/4155253111 shutdown_connections 2024-04-25T15:01:40.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.724+0000 7fc0da800640 1 -- 172.21.15.19:0/4155253111 wait complete. 2024-04-25T15:01:40.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.725+0000 7fc0da800640 1 Processor -- start 2024-04-25T15:01:40.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.725+0000 7fc0da800640 1 -- start start 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 0x7fc0d419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 0x7fc0d419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 0x7fc0d41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410fb10 con 0x7fc0d4105b80 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410f990 con 0x7fc0d4103fd0 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0da800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410fc90 con 0x7fc0d41051c0 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 0x7fc0d41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 0x7fc0d419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 0x7fc0d41a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43570/0 (socket says 172.21.15.19:43570) 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d9e00640 1 -- 172.21.15.19:0/3161639680 learned_addr learned my addr 172.21.15.19:0/3161639680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d9400640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 0x7fc0d419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 -- 172.21.15.19:0/3161639680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 msgr2=0x7fc0d419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 0x7fc0d419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 -- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 msgr2=0x7fc0d41a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:40.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 0x7fc0d41a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.726+0000 7fc0d8a00640 1 -- 172.21.15.19:0/3161639680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41a1d70 con 0x7fc0d41051c0 2024-04-25T15:01:40.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0d8a00640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 0x7fc0d419a690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005fd0 tx=0x7fc0c000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cd00 con 0x7fc0d41051c0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c00035f0 con 0x7fc0d41051c0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0010670 con 0x7fc0d41051c0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d41a2060 con 0x7fc0d41051c0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.727+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a2600 con 0x7fc0d41051c0 2024-04-25T15:01:40.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.728+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d410b2f0 con 0x7fc0d41051c0 2024-04-25T15:01:40.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.731+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc0c00037a0 con 0x7fc0d41051c0 2024-04-25T15:01:40.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.732+0000 7fc0d2000640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 0x7fc0bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:40.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.732+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc0c00d0140 con 0x7fc0d41051c0 2024-04-25T15:01:40.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.732+0000 7fc0d9400640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 0x7fc0bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:40.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.733+0000 7fc0d9400640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 0x7fc0bc0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4105020 tx=0x7fc0c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:40.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.736+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0c0009510 con 0x7fc0d41051c0 2024-04-25T15:01:40.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: Deploying daemon mds.i on smithi102 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6836/3676268156,v1:172.21.15.83:6837/3676268156] up:boot 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: fsmap 8 up:standby 2024-04-25T15:01:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:40 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:01:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: Deploying daemon mds.i on smithi102 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6836/3676268156,v1:172.21.15.83:6837/3676268156] up:boot 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: fsmap 8 up:standby 2024-04-25T15:01:40.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:40 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:01:40.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.992+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d402d5c0 con 0x7fc0d41051c0 2024-04-25T15:01:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.993+0000 7fc0d2000640 1 -- 172.21.15.19:0/3161639680 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7fc0c009cf20 con 0x7fc0d41051c0 2024-04-25T15:01:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T15:01:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T15:01:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 msgr2=0x7fc0bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 0x7fc0bc0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4105020 tx=0x7fc0c8009500 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 msgr2=0x7fc0d419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 0x7fc0d419a690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005fd0 tx=0x7fc0c000ee50 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 shutdown_connections 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d4105b80 0x7fc0d41a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0d41051c0 0x7fc0d419a690 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005fd0 tx=0x7fc0c000ee50 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc0bc0a3c30 0x7fc0bc0a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 --2- 172.21.15.19:0/3161639680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d4103fd0 0x7fc0d419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.995+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 >> 172.21.15.19:0/3161639680 conn(0x7fc0d40ff560 msgr2=0x7fc0d41084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.996+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 shutdown_connections 2024-04-25T15:01:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:40.996+0000 7fc0da800640 1 -- 172.21.15.19:0/3161639680 wait complete. 2024-04-25T15:01:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:41 smithi102 ceph-mon[33005]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:41 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3161639680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:41 smithi083 ceph-mon[32919]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:41 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3161639680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:41.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:41 smithi019 ceph-mon[28090]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:41.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:41 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3161639680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:41.972 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:41 smithi102 systemd[1]: Starting Ceph mds.i for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:42.274 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:41 smithi102 podman[64891]: 2024-04-25 15:01:41.971449879 +0000 UTC m=+0.107251713 container create 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:41 smithi102 podman[64891]: 2024-04-25 15:01:41.882808189 +0000 UTC m=+0.018610039 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 podman[64891]: 2024-04-25 15:01:42.190609991 +0000 UTC m=+0.326411825 container init 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 podman[64891]: 2024-04-25 15:01:42.193902103 +0000 UTC m=+0.329703946 container start 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 ceph-mds[64911]: -- 172.21.15.102:0/3820327703 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b729edfa40 con 0x55b729eb2c00 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i[64907]: starting mds.i at 2024-04-25T15:01:42.275 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 bash[64891]: 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 2024-04-25T15:01:42.557 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:01:42 smithi102 systemd[1]: Started Ceph mds.i for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:43.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 -- 172.21.15.19:0/4232916145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff780077b50 msgr2=0x7ff780077fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 --2- 172.21.15.19:0/4232916145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff780077b50 0x7ff780077fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009900 tx=0x7ff76c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 -- 172.21.15.19:0/4232916145 shutdown_connections 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 --2- 172.21.15.19:0/4232916145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7800784f0 0x7ff780078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 --2- 172.21.15.19:0/4232916145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff780077b50 0x7ff780077fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 --2- 172.21.15.19:0/4232916145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780076910 0x7ff780076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.213+0000 7ff786c00640 1 -- 172.21.15.19:0/4232916145 >> 172.21.15.19:0/4232916145 conn(0x7ff7800ff740 msgr2=0x7ff780101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.214+0000 7ff786c00640 1 -- 172.21.15.19:0/4232916145 shutdown_connections 2024-04-25T15:01:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.214+0000 7ff786c00640 1 -- 172.21.15.19:0/4232916145 wait complete. 2024-04-25T15:01:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.214+0000 7ff786c00640 1 Processor -- start 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 -- start start 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 0x7ff78019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780077b50 0x7ff78019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff780114130 con 0x7ff7800784f0 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff780113fb0 con 0x7ff780077b50 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff786c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7801142b0 con 0x7ff780076910 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff784c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff77fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 0x7ff78019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff784c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43596/0 (socket says 172.21.15.19:43596) 2024-04-25T15:01:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff784c00640 1 -- 172.21.15.19:0/1073782898 learned_addr learned my addr 172.21.15.19:0/1073782898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.215+0000 7ff77f400640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780077b50 0x7ff78019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 -- 172.21.15.19:0/1073782898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780077b50 msgr2=0x7ff78019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780077b50 0x7ff78019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 -- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 msgr2=0x7ff7801a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 -- 172.21.15.19:0/1073782898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7801a63e0 con 0x7ff780076910 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff784c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77fe00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 0x7ff78019e7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff77000ec80 tx=0x7ff77000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770003b70 con 0x7ff780076910 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7801a66d0 con 0x7ff780076910 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff770003d10 con 0x7ff780076910 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.216+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770010930 con 0x7ff780076910 2024-04-25T15:01:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.217+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7801a6bf0 con 0x7ff780076910 2024-04-25T15:01:43.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.218+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff780110d00 con 0x7ff780076910 2024-04-25T15:01:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.220+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff770004560 con 0x7ff780076910 2024-04-25T15:01:43.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.222+0000 7ff77cc00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 0x7ff7640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:43.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.222+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff770044080 con 0x7ff780076910 2024-04-25T15:01:43.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.222+0000 7ff77f400640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 0x7ff7640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:43.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.223+0000 7ff77f400640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 0x7ff7640a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff76c031500 tx=0x7ff76c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:43.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.223+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff77009d4a0 con 0x7ff780076910 2024-04-25T15:01:43.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:43.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: Deploying daemon mds.g on smithi019 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.102:6838/4066270851,v1:172.21.15.102:6839/4066270851] up:boot 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: fsmap 9 up:standby 2024-04-25T15:01:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:43 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:01:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.494+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff78002d5c0 con 0x7ff780076910 2024-04-25T15:01:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.495+0000 7ff77cc00640 1 -- 172.21.15.19:0/1073782898 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7ff77009cda0 con 0x7ff780076910 2024-04-25T15:01:43.497 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:43.497 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T15:01:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 msgr2=0x7ff7640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 0x7ff7640a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff76c031500 tx=0x7ff76c005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 msgr2=0x7ff78019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 0x7ff78019e7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff77000ec80 tx=0x7ff77000c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 shutdown_connections 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7800784f0 0x7ff7801a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff7640a3bb0 0x7ff7640a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff780077b50 0x7ff78019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 --2- 172.21.15.19:0/1073782898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff780076910 0x7ff78019e7c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:43.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 >> 172.21.15.19:0/1073782898 conn(0x7ff7800ff740 msgr2=0x7ff780100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:43.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 shutdown_connections 2024-04-25T15:01:43.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:43.498+0000 7ff786c00640 1 -- 172.21.15.19:0/1073782898 wait complete. 2024-04-25T15:01:43.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: Deploying daemon mds.g on smithi019 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.102:6838/4066270851,v1:172.21.15.102:6839/4066270851] up:boot 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: fsmap 9 up:standby 2024-04-25T15:01:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:43 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:43.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: Deploying daemon mds.g on smithi019 2024-04-25T15:01:43.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.102:6838/4066270851,v1:172.21.15.102:6839/4066270851] up:boot 2024-04-25T15:01:43.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: fsmap 9 up:standby 2024-04-25T15:01:43.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:43 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:01:44.520 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1073782898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:44.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:44 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1073782898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:44 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1073782898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:45.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:45.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:45 smithi019 ceph-mon[28090]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 -- 172.21.15.19:0/1524375064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078101fa0 msgr2=0x7f10781023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 --2- 172.21.15.19:0/1524375064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078101fa0 0x7f10781023c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1068009860 tx=0x7f106802a340 comp rx=0 tx=0).stop 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 -- 172.21.15.19:0/1524375064 shutdown_connections 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 --2- 172.21.15.19:0/1524375064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078103b50 0x7f107810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 --2- 172.21.15.19:0/1524375064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103190 0x7f1078103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 --2- 172.21.15.19:0/1524375064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078101fa0 0x7f10781023c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 -- 172.21.15.19:0/1524375064 >> 172.21.15.19:0/1524375064 conn(0x7f10780fd530 msgr2=0x7f10780ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.724+0000 7f107e400640 1 -- 172.21.15.19:0/1524375064 shutdown_connections 2024-04-25T15:01:45.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.725+0000 7f107e400640 1 -- 172.21.15.19:0/1524375064 wait complete. 2024-04-25T15:01:45.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.725+0000 7f107e400640 1 Processor -- start 2024-04-25T15:01:45.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 -- start start 2024-04-25T15:01:45.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 0x7f107819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 0x7f107819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103b50 0x7f10781a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f107810daa0 con 0x7f1078101fa0 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f107810d920 con 0x7f1078103190 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f107e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f107810dc20 con 0x7f1078103b50 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f1077400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 0x7f107819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f1076a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 0x7f107819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f1077400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 0x7f107819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41190/0 (socket says 172.21.15.19:41190) 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.726+0000 7f1077400640 1 -- 172.21.15.19:0/194469984 learned_addr learned my addr 172.21.15.19:0/194469984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:45.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1077e00640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103b50 0x7f10781a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 -- 172.21.15.19:0/194469984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103b50 msgr2=0x7f10781a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103b50 0x7f10781a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 -- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 msgr2=0x7f107819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 0x7f107819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 -- 172.21.15.19:0/194469984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10781a2040 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f1076a00640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 0x7f107819a960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f106400e9f0 tx=0x7f106400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.727+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106400cd80 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.728+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10640035f0 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.728+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10781a2330 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.728+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064010690 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.728+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781a2880 con 0x7f1078103190 2024-04-25T15:01:45.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.729+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10781a2b40 con 0x7f1078103190 2024-04-25T15:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.731+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10640037a0 con 0x7f1078103190 2024-04-25T15:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.733+0000 7f106fe00640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.733+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f10640d02b0 con 0x7f1078103190 2024-04-25T15:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.733+0000 7f1077400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.734+0000 7f1077400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 0x7f105c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10680095f0 tx=0x7f1068030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:45.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.736+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f106409d090 con 0x7f1078103190 2024-04-25T15:01:45.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:45 smithi083 ceph-mon[32919]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:45 smithi102 ceph-mon[33005]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:45.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.997+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10781a2d10 con 0x7f1078103190 2024-04-25T15:01:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:45.998+0000 7f106fe00640 1 -- 172.21.15.19:0/194469984 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f106409c990 con 0x7f1078103190 2024-04-25T15:01:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:45.999 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T15:01:46.000 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 msgr2=0x7f105c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 0x7f105c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10680095f0 tx=0x7f1068030450 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 msgr2=0x7f107819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 0x7f107819a960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f106400e9f0 tx=0x7f106400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 shutdown_connections 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1078103b50 0x7f10781a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1078103190 0x7f107819a960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 --2- 172.21.15.19:0/194469984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1078101fa0 0x7f107819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.000+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 >> 172.21.15.19:0/194469984 conn(0x7f10780fd530 msgr2=0x7f1078074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:46.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.001+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 shutdown_connections 2024-04-25T15:01:46.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:46.001+0000 7f107e400640 1 -- 172.21.15.19:0/194469984 wait complete. 2024-04-25T15:01:46.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/194469984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:46.622 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:01:46 smithi019 systemd[1]: Started Ceph mds.g for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/194469984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/194469984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:47.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:47.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: Deploying daemon mds.a on smithi019 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.19:6838/2415265237,v1:172.21.15.19:6839/2415265237] up:boot 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: fsmap 10 up:standby 2024-04-25T15:01:47.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:47 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:01:48.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:48.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: Deploying daemon mds.a on smithi019 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.19:6838/2415265237,v1:172.21.15.19:6839/2415265237] up:boot 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: fsmap 10 up:standby 2024-04-25T15:01:48.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:47 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:48.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: Deploying daemon mds.a on smithi019 2024-04-25T15:01:48.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.19:6838/2415265237,v1:172.21.15.19:6839/2415265237] up:boot 2024-04-25T15:01:48.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: fsmap 10 up:standby 2024-04-25T15:01:48.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:47 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:01:48.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.228+0000 7fba0a000640 1 -- 172.21.15.19:0/1983818380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba04101df0 msgr2=0x7fba04102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:48.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.228+0000 7fba0a000640 1 --2- 172.21.15.19:0/1983818380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba04101df0 0x7fba04102210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009ca0 tx=0x7fb9f402a8c0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 -- 172.21.15.19:0/1983818380 shutdown_connections 2024-04-25T15:01:48.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 --2- 172.21.15.19:0/1983818380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba041039a0 0x7fba0410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 --2- 172.21.15.19:0/1983818380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04102fe0 0x7fba04103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 --2- 172.21.15.19:0/1983818380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba04101df0 0x7fba04102210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 -- 172.21.15.19:0/1983818380 >> 172.21.15.19:0/1983818380 conn(0x7fba040fd360 msgr2=0x7fba040ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:48.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 -- 172.21.15.19:0/1983818380 shutdown_connections 2024-04-25T15:01:48.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.229+0000 7fba0a000640 1 -- 172.21.15.19:0/1983818380 wait complete. 2024-04-25T15:01:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 Processor -- start 2024-04-25T15:01:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 -- start start 2024-04-25T15:01:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 0x7fba0419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:48.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 0x7fba0419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba0410d970 con 0x7fba041039a0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba0410d7f0 con 0x7fba04101df0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba0a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba0410daf0 con 0x7fba04102fe0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba09600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.230+0000 7fba09600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41226/0 (socket says 172.21.15.19:41226) 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 -- 172.21.15.19:0/3810818102 learned_addr learned my addr 172.21.15.19:0/3810818102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba08c00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 0x7fba0419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba03e00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 0x7fba0419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 -- 172.21.15.19:0/3810818102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 msgr2=0x7fba0419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 0x7fba0419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 -- 172.21.15.19:0/3810818102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 msgr2=0x7fba0419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:48.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 0x7fba0419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 -- 172.21.15.19:0/3810818102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba041a1e50 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba03e00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 0x7fba0419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba08c00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 0x7fba0419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba09600640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e970 tx=0x7fb9fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.231+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00ccc0 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.232+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba041a2140 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.232+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9fc00ce60 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.232+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc010430 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.232+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba041a2690 con 0x7fba041039a0 2024-04-25T15:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.233+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba041a2950 con 0x7fba041039a0 2024-04-25T15:01:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.235+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9fc004f90 con 0x7fba041039a0 2024-04-25T15:01:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.236+0000 7fba01600640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 0x7fb9e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.237+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb9fc0d0480 con 0x7fba041039a0 2024-04-25T15:01:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.237+0000 7fba08c00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 0x7fb9e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:48.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.238+0000 7fba08c00640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 0x7fb9e80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000f80 tx=0x7fb9f4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.241+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9fc09d260 con 0x7fba041039a0 2024-04-25T15:01:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.510+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba041a2b20 con 0x7fba041039a0 2024-04-25T15:01:48.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.511+0000 7fba01600640 1 -- 172.21.15.19:0/3810818102 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7fb9fc09cb60 con 0x7fba041039a0 2024-04-25T15:01:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:48.513 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T15:01:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 msgr2=0x7fb9e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 0x7fb9e80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000f80 tx=0x7fb9f4004a40 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 msgr2=0x7fba041a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e970 tx=0x7fb9fc00ed70 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 shutdown_connections 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba041039a0 0x7fba041a17e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04102fe0 0x7fba0419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb9e80a3bb0 0x7fb9e80a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 --2- 172.21.15.19:0/3810818102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba04101df0 0x7fba0419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 >> 172.21.15.19:0/3810818102 conn(0x7fba040fd360 msgr2=0x7fba04074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 shutdown_connections 2024-04-25T15:01:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:48.514+0000 7fba0a000640 1 -- 172.21.15.19:0/3810818102 wait complete. 2024-04-25T15:01:48.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:48 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3810818102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:49.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:48 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3810818102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:48 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3810818102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:49.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:49 smithi019 ceph-mon[28090]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:49.937 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:49 smithi102 ceph-mon[33005]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:49.942 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:49 smithi083 ceph-mon[32919]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:50.121 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:49 smithi019 systemd[1]: Starting Ceph mds.a for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:50.532 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 podman[69448]: 2024-04-25 15:01:50.101787623 +0000 UTC m=+0.039806893 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:50.532 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 podman[69448]: 2024-04-25 15:01:50.252074121 +0000 UTC m=+0.190093384 container create 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T15:01:50.532 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 podman[69448]: 2024-04-25 15:01:50.530419273 +0000 UTC m=+0.468438552 container init 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T15:01:50.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 -- 172.21.15.19:0/1273051168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610074930 msgr2=0x7fa610074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 --2- 172.21.15.19:0/1273051168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610074930 0x7fa610074d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009900 tx=0x7fa5fc02a480 comp rx=0 tx=0).stop 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 -- 172.21.15.19:0/1273051168 shutdown_connections 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 --2- 172.21.15.19:0/1273051168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610109e50 0x7fa61010c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 --2- 172.21.15.19:0/1273051168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610074930 0x7fa610074d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 --2- 172.21.15.19:0/1273051168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610075f10 0x7fa610074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 -- 172.21.15.19:0/1273051168 >> 172.21.15.19:0/1273051168 conn(0x7fa6100fd530 msgr2=0x7fa6100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 -- 172.21.15.19:0/1273051168 shutdown_connections 2024-04-25T15:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.750+0000 7fa618600640 1 -- 172.21.15.19:0/1273051168 wait complete. 2024-04-25T15:01:50.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.751+0000 7fa618600640 1 Processor -- start 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.751+0000 7fa618600640 1 -- start start 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610075f10 0x7fa61019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610109e50 0x7fa6101a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010fd40 con 0x7fa610109e50 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010fbc0 con 0x7fa610075f10 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa618600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010fec0 con 0x7fa610074930 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:55502/0 (socket says 172.21.15.19:55502) 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610075f10 0x7fa61019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 -- 172.21.15.19:0/3030851387 learned_addr learned my addr 172.21.15.19:0/3030851387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa616600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610109e50 0x7fa6101a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 -- 172.21.15.19:0/3030851387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610075f10 msgr2=0x7fa61019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:50.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610075f10 0x7fa61019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 -- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610109e50 msgr2=0x7fa6101a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610109e50 0x7fa6101a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.752+0000 7fa615c00640 1 -- 172.21.15.19:0/3030851387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a1fb0 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa615c00640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ebc0 tx=0x7fa60000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600010630 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600010c80 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600018b60 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a22a0 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.753+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a27c0 con 0x7fa610074930 2024-04-25T15:01:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.754+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa610108c10 con 0x7fa610074930 2024-04-25T15:01:50.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.756+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa600018d00 con 0x7fa610074930 2024-04-25T15:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.758+0000 7fa60e000640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 0x7fa5f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.758+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa6000d05d0 con 0x7fa610074930 2024-04-25T15:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.758+0000 7fa615200640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 0x7fa5f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.759+0000 7fa615200640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 0x7fa5f40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc031500 tx=0x7fa5fc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:50.762+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa60009cd60 con 0x7fa610074930 2024-04-25T15:01:50.872 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 podman[69448]: 2024-04-25 15:01:50.541014887 +0000 UTC m=+0.479034118 container start 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T15:01:50.873 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 ceph-mds[69467]: -- 172.21.15.19:0/1549678810 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55614b607a40 con 0x55614b5da800 2024-04-25T15:01:50.873 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: starting mds.a at 2024-04-25T15:01:50.873 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 bash[69448]: 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d 2024-04-25T15:01:50.873 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:01:50 smithi019 systemd[1]: Started Ceph mds.a for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.016+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa610062b50 con 0x7fa610074930 2024-04-25T15:01:51.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.018+0000 7fa60e000640 1 -- 172.21.15.19:0/3030851387 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7fa6000a1060 con 0x7fa610074930 2024-04-25T15:01:51.099 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:51.100 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 msgr2=0x7fa5f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 0x7fa5f40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc031500 tx=0x7fa5fc005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 msgr2=0x7fa61019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ebc0 tx=0x7fa60000c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 shutdown_connections 2024-04-25T15:01:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610109e50 0x7fa6101a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5f40a3bb0 0x7fa5f40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa610075f10 0x7fa61019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 --2- 172.21.15.19:0/3030851387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa610074930 0x7fa61019a390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 >> 172.21.15.19:0/3030851387 conn(0x7fa6100fd530 msgr2=0x7fa610105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 shutdown_connections 2024-04-25T15:01:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:51.021+0000 7fa618600640 1 -- 172.21.15.19:0/3030851387 wait complete. 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: Deploying daemon mds.b on smithi083 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3030851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.19:6840/2613932433,v1:172.21.15.19:6841/2613932433] up:boot 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: fsmap 11 up:standby 2024-04-25T15:01:52.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:51 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: Deploying daemon mds.b on smithi083 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3030851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.19:6840/2613932433,v1:172.21.15.19:6841/2613932433] up:boot 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: fsmap 11 up:standby 2024-04-25T15:01:52.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:51 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: Deploying daemon mds.b on smithi083 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3030851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.19:6840/2613932433,v1:172.21.15.19:6841/2613932433] up:boot 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: fsmap 11 up:standby 2024-04-25T15:01:52.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:51 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:01:53.041 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:52 smithi083 systemd[1]: Starting Ceph mds.b for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:01:53.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.319+0000 7f143ea00640 1 -- 172.21.15.19:0/286916046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 msgr2=0x7f14381087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.319+0000 7f143ea00640 1 --2- 172.21.15.19:0/286916046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f14381087f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f142c009900 tx=0x7f142c02a410 comp rx=0 tx=0).stop 2024-04-25T15:01:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 -- 172.21.15.19:0/286916046 shutdown_connections 2024-04-25T15:01:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 --2- 172.21.15.19:0/286916046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 0x7f1438110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 --2- 172.21.15.19:0/286916046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f1438109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 --2- 172.21.15.19:0/286916046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f14381087f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 -- 172.21.15.19:0/286916046 >> 172.21.15.19:0/286916046 conn(0x7f1438077390 msgr2=0x7f14380797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 -- 172.21.15.19:0/286916046 shutdown_connections 2024-04-25T15:01:53.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.320+0000 7f143ea00640 1 -- 172.21.15.19:0/286916046 wait complete. 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.321+0000 7f143ea00640 1 Processor -- start 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.321+0000 7f143ea00640 1 -- start start 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 0x7f14381a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:53.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1438113f60 con 0x7f14381095c0 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1438113de0 con 0x7f14381083d0 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f14381140e0 con 0x7f1438109f80 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:35340/0 (socket says 172.21.15.19:35340) 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41284/0 (socket says 172.21.15.19:41284) 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143cc00640 1 -- 172.21.15.19:0/1198050612 learned_addr learned my addr 172.21.15.19:0/1198050612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.322+0000 7f143e000640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 0x7f14381a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 -- 172.21.15.19:0/1198050612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 msgr2=0x7f14381a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 0x7f14381a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 -- 172.21.15.19:0/1198050612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 msgr2=0x7f143819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 -- 172.21.15.19:0/1198050612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14381a6200 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143d600640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143cc00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f142400ea60 tx=0x7f142400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142400cde0 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14240035f0 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14381a64f0 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.323+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424010690 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.324+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14381a6a40 con 0x7f14381095c0 2024-04-25T15:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.325+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143810f790 con 0x7f14381095c0 2024-04-25T15:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.327+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14240037a0 con 0x7f14381095c0 2024-04-25T15:01:53.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.330+0000 7f1436000640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 0x7f14200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.330+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f14240d0290 con 0x7f14381095c0 2024-04-25T15:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.330+0000 7f143d600640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 0x7f14200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.331+0000 7f143d600640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 0x7f14200a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f142c005ab0 tx=0x7f142c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.335+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f142409d070 con 0x7f14381095c0 2024-04-25T15:01:53.390 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 podman[63142]: 2024-04-25 15:01:53.02478183 +0000 UTC m=+0.018222163 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:01:53.390 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 podman[63142]: 2024-04-25 15:01:53.144826168 +0000 UTC m=+0.138266497 container create cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:01:53.390 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 podman[63142]: 2024-04-25 15:01:53.389487978 +0000 UTC m=+0.382928301 container init cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:01:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.590+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f143802d5c0 con 0x7f14381095c0 2024-04-25T15:01:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.596+0000 7f1436000640 1 -- 172.21.15.19:0/1198050612 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f142409c970 con 0x7f14381095c0 2024-04-25T15:01:53.597 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:53.597 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-25T15:01:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-04-25T15:01:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 msgr2=0x7f14200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 0x7f14200a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f142c005ab0 tx=0x7f142c005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 msgr2=0x7f143819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f142400ea60 tx=0x7f142400ef20 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 shutdown_connections 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1438109f80 0x7f14381a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14381095c0 0x7f143819eb20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f14200a3bb0 0x7f14200a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 --2- 172.21.15.19:0/1198050612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14381083d0 0x7f143819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 >> 172.21.15.19:0/1198050612 conn(0x7f1438077390 msgr2=0x7f143810c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 shutdown_connections 2024-04-25T15:01:53.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:53.599+0000 7f143ea00640 1 -- 172.21.15.19:0/1198050612 wait complete. 2024-04-25T15:01:53.744 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 podman[63142]: 2024-04-25 15:01:53.392193959 +0000 UTC m=+0.385634283 container start cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T15:01:53.744 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mds[63161]: -- 172.21.15.83:0/1262705681 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c61414fa40 con 0x55c614122800 2024-04-25T15:01:53.744 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: starting mds.b at 2024-04-25T15:01:53.744 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 bash[63142]: cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a 2024-04-25T15:01:53.744 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:01:53 smithi083 systemd[1]: Started Ceph mds.b for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:01:53.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1198050612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:53.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:53 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1198050612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:53 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1198050612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:53 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:55.034 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:54 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:boot 2024-04-25T15:01:55.034 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:54 smithi083 ceph-mon[32919]: fsmap 12 up:standby 2024-04-25T15:01:55.035 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:54 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:01:55.035 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:54 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:54 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:boot 2024-04-25T15:01:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:54 smithi102 ceph-mon[33005]: fsmap 12 up:standby 2024-04-25T15:01:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:54 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:01:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:54 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:54 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:boot 2024-04-25T15:01:55.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:54 smithi019 ceph-mon[28090]: fsmap 12 up:standby 2024-04-25T15:01:55.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:54 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:01:55.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:54 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:55.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 -- 172.21.15.19:0/2283365924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 msgr2=0x7febcc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 --2- 172.21.15.19:0/2283365924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc1043f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febbc009900 tx=0x7febbc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 -- 172.21.15.19:0/2283365924 shutdown_connections 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 --2- 172.21.15.19:0/2283365924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc10c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 --2- 172.21.15.19:0/2283365924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 0x7febcc105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 --2- 172.21.15.19:0/2283365924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc1043f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 -- 172.21.15.19:0/2283365924 >> 172.21.15.19:0/2283365924 conn(0x7febcc0ff560 msgr2=0x7febcc101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 -- 172.21.15.19:0/2283365924 shutdown_connections 2024-04-25T15:01:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.862+0000 7febd0e00640 1 -- 172.21.15.19:0/2283365924 wait complete. 2024-04-25T15:01:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.863+0000 7febd0e00640 1 Processor -- start 2024-04-25T15:01:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.863+0000 7febd0e00640 1 -- start start 2024-04-25T15:01:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 0x7febcc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10fb20 con 0x7febcc105b80 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10f9a0 con 0x7febcc1051c0 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febd0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10fca0 con 0x7febcc103fd0 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:33692/0 (socket says 172.21.15.19:33692) 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 -- 172.21.15.19:0/3539777140 learned_addr learned my addr 172.21.15.19:0/3539777140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44270/0 (socket says 172.21.15.19:44270) 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcaa00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 0x7febcc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 -- 172.21.15.19:0/3539777140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 msgr2=0x7febcc19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 0x7febcc19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 -- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 msgr2=0x7febcc1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.864+0000 7febcb400640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febcb400640 1 -- 172.21.15.19:0/3539777140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc1a1e70 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febcb400640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7febbc031500 tx=0x7febbc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc03f070 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1a2100 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febbc031c50 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc035030 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.865+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1a2640 con 0x7febcc103fd0 2024-04-25T15:01:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.867+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febcc10b360 con 0x7febcc103fd0 2024-04-25T15:01:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.869+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7febbc042070 con 0x7febcc103fd0 2024-04-25T15:01:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.871+0000 7febc3e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 0x7febb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.871+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7febbc0f0ed0 con 0x7febcc103fd0 2024-04-25T15:01:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.871+0000 7febcaa00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 0x7febb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.872+0000 7febcaa00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 0x7febb00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febcc19b760 tx=0x7febb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:55.872+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febbc0bdcb0 con 0x7febcc103fd0 2024-04-25T15:01:56.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:55 smithi083 ceph-mon[32919]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:55 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:55 smithi102 ceph-mon[33005]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:55 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:55 smithi019 ceph-mon[28090]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:55 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:56.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.177+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febcc02d810 con 0x7febcc103fd0 2024-04-25T15:01:56.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.178+0000 7febc3e00640 1 -- 172.21.15.19:0/3539777140 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7febbc0bd5b0 con 0x7febcc103fd0 2024-04-25T15:01:56.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:56.180 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T15:01:56.180 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 msgr2=0x7febb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 0x7febb00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febcc19b760 tx=0x7febb8005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 msgr2=0x7febcc19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7febbc031500 tx=0x7febbc003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 shutdown_connections 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febcc105b80 0x7febcc1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7febb00a3bb0 0x7febb00a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febcc1051c0 0x7febcc19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 --2- 172.21.15.19:0/3539777140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc103fd0 0x7febcc19a250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 >> 172.21.15.19:0/3539777140 conn(0x7febcc0ff560 msgr2=0x7febcc108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 shutdown_connections 2024-04-25T15:01:56.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.181+0000 7febd0e00640 1 -- 172.21.15.19:0/3539777140 wait complete. 2024-04-25T15:01:56.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 -- 172.21.15.19:0/1087126331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00109e30 msgr2=0x7f2a0010c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 --2- 172.21.15.19:0/1087126331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00109e30 0x7f2a0010c2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009be0 tx=0x7f29f802a750 comp rx=0 tx=0).stop 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 -- 172.21.15.19:0/1087126331 shutdown_connections 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 --2- 172.21.15.19:0/1087126331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00109e30 0x7f2a0010c2d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 --2- 172.21.15.19:0/1087126331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00074950 0x7f2a00074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 --2- 172.21.15.19:0/1087126331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00075f30 0x7f2a00074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 -- 172.21.15.19:0/1087126331 >> 172.21.15.19:0/1087126331 conn(0x7f2a000fd550 msgr2=0x7f2a000ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 -- 172.21.15.19:0/1087126331 shutdown_connections 2024-04-25T15:01:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.396+0000 7f2a07000640 1 -- 172.21.15.19:0/1087126331 wait complete. 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.397+0000 7f2a07000640 1 Processor -- start 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 -- start start 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 0x7f2a00105320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 0x7f2a00105860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 0x7f2a00101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0010fce0 con 0x7f2a00075f30 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0010fb60 con 0x7f2a00109e30 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a07000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0010fe60 con 0x7f2a00074950 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f29ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 0x7f2a00105860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a05000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 0x7f2a00101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f29ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 0x7f2a00105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f29ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 0x7f2a00105860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44278/0 (socket says 172.21.15.19:44278) 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f2a05000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 0x7f2a00101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:46946/0 (socket says 172.21.15.19:46946) 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.398+0000 7f29ff400640 1 -- 172.21.15.19:0/2294215636 learned_addr learned my addr 172.21.15.19:0/2294215636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 -- 172.21.15.19:0/2294215636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 msgr2=0x7f2a00101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 0x7f2a00101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 -- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 msgr2=0x7f2a00105860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 0x7f2a00105860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 -- 172.21.15.19:0/2294215636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a00102750 con 0x7f2a00074950 2024-04-25T15:01:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29ffe00640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 0x7f2a00105320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29f000e790 tx=0x7f29f000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f000eea0 con 0x7f2a00074950 2024-04-25T15:01:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f000ccc0 con 0x7f2a00074950 2024-04-25T15:01:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a001a67a0 con 0x7f2a00074950 2024-04-25T15:01:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0018680 con 0x7f2a00074950 2024-04-25T15:01:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.399+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a001a6bb0 con 0x7f2a00074950 2024-04-25T15:01:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.401+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00108bf0 con 0x7f2a00074950 2024-04-25T15:01:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.403+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29f0005030 con 0x7f2a00074950 2024-04-25T15:01:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.404+0000 7f29fcc00640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 0x7f29e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.404+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f29f00d0190 con 0x7f2a00074950 2024-04-25T15:01:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.404+0000 7f29ff400640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 0x7f29e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.405+0000 7f29ff400640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 0x7f29e40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0095f0 tx=0x7f29ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.406+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29f009c920 con 0x7f2a00074950 2024-04-25T15:01:56.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.642+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a00062c50 con 0x7f2a00074950 2024-04-25T15:01:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.643+0000 7f29fcc00640 1 -- 172.21.15.19:0/2294215636 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18385 (secure 0 0 0) 0x7f29f00a0c20 con 0x7f2a00074950 2024-04-25T15:01:56.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:56.645 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":84,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:01:03.497686+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 msgr2=0x7f29e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 0x7f29e40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0095f0 tx=0x7f29ec005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 msgr2=0x7f2a00105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 0x7f2a00105320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29f000e790 tx=0x7f29f000ec50 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 shutdown_connections 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29e40a3bb0 0x7f29e40a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a00109e30 0x7f2a00101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a00075f30 0x7f2a00105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 --2- 172.21.15.19:0/2294215636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a00074950 0x7f2a00105320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 >> 172.21.15.19:0/2294215636 conn(0x7f2a000fd550 msgr2=0x7f2a00105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 shutdown_connections 2024-04-25T15:01:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.646+0000 7f2a07000640 1 -- 172.21.15.19:0/2294215636 wait complete. 2024-04-25T15:01:56.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:56.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 -- 172.21.15.19:0/805811244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481019d0 msgr2=0x7f7548101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 --2- 172.21.15.19:0/805811244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481019d0 0x7f7548101e50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7540009ca0 tx=0x7f754002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 -- 172.21.15.19:0/805811244 shutdown_connections 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 --2- 172.21.15.19:0/805811244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481019d0 0x7f7548101e50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 --2- 172.21.15.19:0/805811244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7548101030 0x7f7548101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 --2- 172.21.15.19:0/805811244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75481006d0 0x7f7548100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 -- 172.21.15.19:0/805811244 >> 172.21.15.19:0/805811244 conn(0x7f75480fc2f0 msgr2=0x7f75480fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 -- 172.21.15.19:0/805811244 shutdown_connections 2024-04-25T15:01:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.864+0000 7f7547200640 1 -- 172.21.15.19:0/805811244 wait complete. 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.865+0000 7f7547200640 1 Processor -- start 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 -- start start 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7548101030 0x7f754819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481019d0 0x7f75481a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f754810fc00 con 0x7f75481006d0 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f754810fa80 con 0x7f7548101030 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7547200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f754810fd80 con 0x7f75481019d0 2024-04-25T15:01:56.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7545e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7545e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44306/0 (socket says 172.21.15.19:44306) 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7545e00640 1 -- 172.21.15.19:0/1435062434 learned_addr learned my addr 172.21.15.19:0/1435062434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7546800640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481019d0 0x7f75481a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.866+0000 7f7545400640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7548101030 0x7f754819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 -- 172.21.15.19:0/1435062434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481019d0 msgr2=0x7f75481a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481019d0 0x7f75481a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 -- 172.21.15.19:0/1435062434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7548101030 msgr2=0x7f754819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7548101030 0x7f754819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 -- 172.21.15.19:0/1435062434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75481a1e60 con 0x7f75481006d0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7545e00640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f753800e9f0 tx=0x7f753800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753800cd80 con 0x7f75481006d0 2024-04-25T15:01:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75380108e0 con 0x7f75481006d0 2024-04-25T15:01:56.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538018980 con 0x7f75481006d0 2024-04-25T15:01:56.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.867+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75481a2150 con 0x7f75481006d0 2024-04-25T15:01:56.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.868+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a26a0 con 0x7f75481006d0 2024-04-25T15:01:56.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.869+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754810b420 con 0x7f75481006d0 2024-04-25T15:01:56.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.871+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7538010430 con 0x7f75481006d0 2024-04-25T15:01:56.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.872+0000 7f753ea00640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 0x7f752c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.873+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f75380d02d0 con 0x7f75481006d0 2024-04-25T15:01:56.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.873+0000 7f7545400640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 0x7f752c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.874+0000 7f7545400640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 0x7f752c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f754819b750 tx=0x7f7534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:56.878+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f753809ca60 con 0x7f75481006d0 2024-04-25T15:01:57.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.101+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7548062b50 con 0x7f75481006d0 2024-04-25T15:01:57.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.102+0000 7f753ea00640 1 -- 172.21.15.19:0/1435062434 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f75380a0d60 con 0x7f75481006d0 2024-04-25T15:01:57.103 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 msgr2=0x7f752c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 0x7f752c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f754819b750 tx=0x7f7534005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 msgr2=0x7f754819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f753800e9f0 tx=0x7f753800eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 shutdown_connections 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481019d0 0x7f75481a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f752c0a3bb0 0x7f752c0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7548101030 0x7f754819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 --2- 172.21.15.19:0/1435062434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75481006d0 0x7f754819a240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 >> 172.21.15.19:0/1435062434 conn(0x7f75480fc2f0 msgr2=0x7f75480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 shutdown_connections 2024-04-25T15:01:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.105+0000 7f7547200640 1 -- 172.21.15.19:0/1435062434 wait complete. 2024-04-25T15:01:57.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T15:01:57.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3539777140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2294215636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:57.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3539777140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2294215636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:57.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 -- 172.21.15.19:0/3927797916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0074930 msgr2=0x7f18d0074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3927797916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0074930 0x7f18d0074d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009900 tx=0x7f18bc02a480 comp rx=0 tx=0).stop 2024-04-25T15:01:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 -- 172.21.15.19:0/3927797916 shutdown_connections 2024-04-25T15:01:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3927797916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0109e50 0x7f18d010c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3927797916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0074930 0x7f18d0074d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3927797916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0075f10 0x7f18d0074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 -- 172.21.15.19:0/3927797916 >> 172.21.15.19:0/3927797916 conn(0x7f18d00fd530 msgr2=0x7f18d00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 -- 172.21.15.19:0/3927797916 shutdown_connections 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.306+0000 7f18d5e00640 1 -- 172.21.15.19:0/3927797916 wait complete. 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.307+0000 7f18d5e00640 1 Processor -- start 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 -- start start 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0074930 0x7f18d019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0109e50 0x7f18d01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010fd10 con 0x7f18d0074930 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010fb90 con 0x7f18d0075f10 2024-04-25T15:01:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18d5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010fe90 con 0x7f18d0109e50 2024-04-25T15:01:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18cea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0074930 0x7f18d019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0109e50 0x7f18d01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:46968/0 (socket says 172.21.15.19:46968) 2024-04-25T15:01:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.308+0000 7f18ce000640 1 -- 172.21.15.19:0/3846824993 learned_addr learned my addr 172.21.15.19:0/3846824993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 -- 172.21.15.19:0/3846824993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0109e50 msgr2=0x7f18d01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0109e50 0x7f18d01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 -- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0074930 msgr2=0x7f18d019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0074930 0x7f18d019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 -- 172.21.15.19:0/3846824993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a1fd0 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18ce000640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005ab0 tx=0x7f18bc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc03f070 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18bc031c50 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a2260 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.309+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc035030 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.310+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a27a0 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.311+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d0108c10 con 0x7f18d0075f10 2024-04-25T15:01:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.313+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18bc042070 con 0x7f18d0075f10 2024-04-25T15:01:57.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.314+0000 7f18d4a00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 0x7f18b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.314+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f18bc0f06a0 con 0x7f18d0075f10 2024-04-25T15:01:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.315+0000 7f18cea00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 0x7f18b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.315+0000 7f18cea00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 0x7f18b40a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18c00095f0 tx=0x7f18c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.319+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18bc0bce30 con 0x7f18d0075f10 2024-04-25T15:01:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3539777140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2294215636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:57.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.529+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f18d002d5c0 con 0x7f18d0075f10 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.530+0000 7f18d4a00640 1 -- 172.21.15.19:0/3846824993 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f18bc0c1130 con 0x7f18d0075f10 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:01:57.531 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T15:01:57.532 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T15:01:57.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:01:57.532 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T15:01:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.532+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 msgr2=0x7f18b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.532+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 0x7f18b40a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18c00095f0 tx=0x7f18c0005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.532+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 msgr2=0x7f18d019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.532+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005ab0 tx=0x7f18bc003b90 comp rx=0 tx=0).stop 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 shutdown_connections 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d0109e50 0x7f18d01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f18b40a3c30 0x7f18b40a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0075f10 0x7f18d019a8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 --2- 172.21.15.19:0/3846824993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18d0074930 0x7f18d019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 >> 172.21.15.19:0/3846824993 conn(0x7f18d00fd530 msgr2=0x7f18d0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 shutdown_connections 2024-04-25T15:01:57.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.533+0000 7f18d5e00640 1 -- 172.21.15.19:0/3846824993 wait complete. 2024-04-25T15:01:57.554 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T15:01:57.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T15:01:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 -- 172.21.15.19:0/1144307098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98103b30 msgr2=0x7f3c9810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 --2- 172.21.15.19:0/1144307098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98103b30 0x7f3c9810a3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009ca0 tx=0x7f3c9002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 -- 172.21.15.19:0/1144307098 shutdown_connections 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 --2- 172.21.15.19:0/1144307098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98103b30 0x7f3c9810a3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 --2- 172.21.15.19:0/1144307098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c981035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 --2- 172.21.15.19:0/1144307098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98101f80 0x7f3c981023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 -- 172.21.15.19:0/1144307098 >> 172.21.15.19:0/1144307098 conn(0x7f3c980fd510 msgr2=0x7f3c980ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 -- 172.21.15.19:0/1144307098 shutdown_connections 2024-04-25T15:01:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.748+0000 7f3c9e200640 1 -- 172.21.15.19:0/1144307098 wait complete. 2024-04-25T15:01:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.749+0000 7f3c9e200640 1 Processor -- start 2024-04-25T15:01:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.749+0000 7f3c9e200640 1 -- start start 2024-04-25T15:01:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 0x7f3c98075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98103b30 0x7f3c98074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810dac0 con 0x7f3c98103170 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810d940 con 0x7f3c98103b30 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c9e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9810dc40 con 0x7f3c98101f80 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44330/0 (socket says 172.21.15.19:44330) 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 -- 172.21.15.19:0/2068333462 learned_addr learned my addr 172.21.15.19:0/2068333462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c97400640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 0x7f3c98075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c97e00640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98103b30 0x7f3c98074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 -- 172.21.15.19:0/2068333462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 msgr2=0x7f3c98075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 0x7f3c98075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 -- 172.21.15.19:0/2068333462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98103b30 msgr2=0x7f3c98074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98103b30 0x7f3c98074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c8ea00640 1 -- 172.21.15.19:0/2068333462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c98074dc0 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.750+0000 7f3c97400640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 0x7f3c98075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c8ea00640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000e9f0 tx=0x7f3c8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8000cd80 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c800108e0 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80018980 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c980750b0 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.751+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981a6c30 con 0x7f3c98103170 2024-04-25T15:01:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.753+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c980691c0 con 0x7f3c98103170 2024-04-25T15:01:57.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.755+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c80010430 con 0x7f3c98103170 2024-04-25T15:01:57.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.756+0000 7f3c94c00640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 0x7f3c7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.756+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f3c800d02c0 con 0x7f3c98103170 2024-04-25T15:01:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.756+0000 7f3c97400640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 0x7f3c7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.757+0000 7f3c97400640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 0x7f3c7c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c98102fd0 tx=0x7f3c88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:57.760+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c8009ca50 con 0x7f3c98103170 2024-04-25T15:01:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:58.003+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3c98062b50 con 0x7f3c98103170 2024-04-25T15:01:58.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:58 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:58.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:58 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1435062434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.290 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:58 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3846824993' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T15:01:58.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:58 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:58.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:58 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1435062434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:58 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3846824993' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T15:01:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:58 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:01:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:58 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1435062434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:58 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3846824993' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T15:01:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.012+0000 7f3c94c00640 1 -- 172.21.15.19:0/2068333462 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f3c800a0d50 con 0x7f3c98103170 2024-04-25T15:01:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T15:01:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 msgr2=0x7f3c7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 0x7f3c7c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c98102fd0 tx=0x7f3c88005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 msgr2=0x7f3c98074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000e9f0 tx=0x7f3c8000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 shutdown_connections 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f3c7c0a3c30 0x7f3c7c0a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c98103b30 0x7f3c98074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c98103170 0x7f3c98074340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 --2- 172.21.15.19:0/2068333462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c98101f80 0x7f3c98075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 >> 172.21.15.19:0/2068333462 conn(0x7f3c980fd510 msgr2=0x7f3c981064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 shutdown_connections 2024-04-25T15:01:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.016+0000 7f3c9e200640 1 -- 172.21.15.19:0/2068333462 wait complete. 2024-04-25T15:01:59.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T15:01:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 -- 172.21.15.19:0/82745272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff304076910 msgr2=0x7ff304076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 --2- 172.21.15.19:0/82745272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff304076910 0x7ff304076d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009900 tx=0x7ff2f802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 -- 172.21.15.19:0/82745272 shutdown_connections 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 --2- 172.21.15.19:0/82745272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3040784f0 0x7ff304078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 --2- 172.21.15.19:0/82745272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304077b50 0x7ff304077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 --2- 172.21.15.19:0/82745272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff304076910 0x7ff304076d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 -- 172.21.15.19:0/82745272 >> 172.21.15.19:0/82745272 conn(0x7ff3040ff740 msgr2=0x7ff304101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 -- 172.21.15.19:0/82745272 shutdown_connections 2024-04-25T15:01:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.228+0000 7ff30be00640 1 -- 172.21.15.19:0/82745272 wait complete. 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.229+0000 7ff30be00640 1 Processor -- start 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.229+0000 7ff30be00640 1 -- start start 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.229+0000 7ff30be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 0x7ff30419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.229+0000 7ff30be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 0x7ff30419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff30be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3040784f0 0x7ff3041a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff30be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114150 con 0x7ff304076910 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff30be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff304113fd0 con 0x7ff304077b50 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff30be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041142d0 con 0x7ff3040784f0 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff309400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 0x7ff30419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 0x7ff30419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff309400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 0x7ff30419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44356/0 (socket says 172.21.15.19:44356) 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff309400640 1 -- 172.21.15.19:0/1102984536 learned_addr learned my addr 172.21.15.19:0/1102984536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff309e00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3040784f0 0x7ff3041a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 -- 172.21.15.19:0/1102984536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3040784f0 msgr2=0x7ff3041a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3040784f0 0x7ff3041a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 -- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 msgr2=0x7ff30419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 0x7ff30419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.230+0000 7ff308a00640 1 -- 172.21.15.19:0/1102984536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3041a6400 con 0x7ff304077b50 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff308a00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 0x7ff30419ed20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e9f0 tx=0x7ff2f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000ccc0 con 0x7ff304077b50 2024-04-25T15:01:59.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3041a66f0 con 0x7ff304077b50 2024-04-25T15:01:59.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f00108e0 con 0x7ff304077b50 2024-04-25T15:01:59.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0018880 con 0x7ff304077b50 2024-04-25T15:01:59.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.231+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a6c40 con 0x7ff304077b50 2024-04-25T15:01:59.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.232+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff304110b40 con 0x7ff304077b50 2024-04-25T15:01:59.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.235+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2f0010430 con 0x7ff304077b50 2024-04-25T15:01:59.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.236+0000 7ff302000640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 0x7ff2ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.236+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7ff2f00d0460 con 0x7ff304077b50 2024-04-25T15:01:59.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.237+0000 7ff309400640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 0x7ff2ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.237+0000 7ff309400640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 0x7ff2ec0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80096c0 tx=0x7ff2f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:59.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.240+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2f009ca80 con 0x7ff304077b50 2024-04-25T15:01:59.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:59.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T15:01:59.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:01:59 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:01:59 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:01:59 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:01:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:01:59.487+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff304062b50 con 0x7ff304077b50 2024-04-25T15:02:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.020+0000 7ff302000640 1 -- 172.21.15.19:0/1102984536 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7ff2f00a0d80 con 0x7ff304077b50 2024-04-25T15:02:00.025 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 msgr2=0x7ff2ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 0x7ff2ec0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80096c0 tx=0x7ff2f8005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 msgr2=0x7ff30419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 0x7ff30419ed20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e9f0 tx=0x7ff2f000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 shutdown_connections 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3040784f0 0x7ff3041a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff2ec0a3bb0 0x7ff2ec0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff304077b50 0x7ff30419ed20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 --2- 172.21.15.19:0/1102984536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff304076910 0x7ff30419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.033+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 >> 172.21.15.19:0/1102984536 conn(0x7ff3040ff740 msgr2=0x7ff304102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.034+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 shutdown_connections 2024-04-25T15:02:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.034+0000 7ff30be00640 1 -- 172.21.15.19:0/1102984536 wait complete. 2024-04-25T15:02:00.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.255+0000 7f2c3a600640 1 -- 172.21.15.19:0/1532762708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34108490 msgr2=0x7f2c341088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 --2- 172.21.15.19:0/1532762708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34108490 0x7f2c341088b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009860 tx=0x7f2c2802a340 comp rx=0 tx=0).stop 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 -- 172.21.15.19:0/1532762708 shutdown_connections 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 --2- 172.21.15.19:0/1532762708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 --2- 172.21.15.19:0/1532762708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34109680 0x7f2c34109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 --2- 172.21.15.19:0/1532762708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34108490 0x7f2c341088b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 -- 172.21.15.19:0/1532762708 >> 172.21.15.19:0/1532762708 conn(0x7f2c34077420 msgr2=0x7f2c34079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 -- 172.21.15.19:0/1532762708 shutdown_connections 2024-04-25T15:02:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.256+0000 7f2c3a600640 1 -- 172.21.15.19:0/1532762708 wait complete. 2024-04-25T15:02:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.257+0000 7f2c3a600640 1 Processor -- start 2024-04-25T15:02:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.257+0000 7f2c3a600640 1 -- start start 2024-04-25T15:02:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.257+0000 7f2c3a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 0x7f2c3419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c3a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c3a600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34113fd0 con 0x7f2c34108490 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c3a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34113e50 con 0x7f2c3410a040 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c3a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34114150 con 0x7f2c34109680 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 0x7f2c3419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:47026/0 (socket says 172.21.15.19:47026) 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 -- 172.21.15.19:0/4049551202 learned_addr learned my addr 172.21.15.19:0/4049551202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44384/0 (socket says 172.21.15.19:44384) 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 -- 172.21.15.19:0/4049551202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 msgr2=0x7f2c3419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 0x7f2c3419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 -- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 msgr2=0x7f2c3419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39c00640 1 -- 172.21.15.19:0/4049551202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c341a62d0 con 0x7f2c3410a040 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c33e00640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 0x7f2c3419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.258+0000 7f2c39200640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c39c00640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00eb10 tx=0x7f2c2c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004b20 con 0x7f2c3410a040 2024-04-25T15:02:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c341a65c0 con 0x7f2c3410a040 2024-04-25T15:02:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c004cc0 con 0x7f2c3410a040 2024-04-25T15:02:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c010690 con 0x7f2c3410a040 2024-04-25T15:02:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.259+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c341a6b10 con 0x7f2c3410a040 2024-04-25T15:02:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.260+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3410f880 con 0x7f2c3410a040 2024-04-25T15:02:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.262+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c2c005020 con 0x7f2c3410a040 2024-04-25T15:02:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.264+0000 7f2c31600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 0x7f2c1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.264+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7f2c2c0d07f0 con 0x7f2c3410a040 2024-04-25T15:02:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.264+0000 7f2c39200640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 0x7f2c1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.265+0000 7f2c39200640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 0x7f2c1c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009830 tx=0x7f2c28030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:00.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.266+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c2c09d2e0 con 0x7f2c3410a040 2024-04-25T15:02:00.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T15:02:00.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T15:02:00.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1102984536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:00.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1102984536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2068333462' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1102984536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:00.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:02:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:00.516+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2c3402d5c0 con 0x7f2c3410a040 2024-04-25T15:02:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.047+0000 7f2c31600640 1 -- 172.21.15.19:0/4049551202 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2c2c09cbe0 con 0x7f2c3410a040 2024-04-25T15:02:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T15:02:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.053+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 msgr2=0x7f2c1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.053+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 0x7f2c1c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009830 tx=0x7f2c28030450 comp rx=0 tx=0).stop 2024-04-25T15:02:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.053+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 msgr2=0x7f2c341a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.053+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00eb10 tx=0x7f2c2c00c2d0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 shutdown_connections 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c1c0a3bb0 0x7f2c1c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c3410a040 0x7f2c341a5c60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c34109680 0x7f2c3419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 --2- 172.21.15.19:0/4049551202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c34108490 0x7f2c3419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 >> 172.21.15.19:0/4049551202 conn(0x7f2c34077420 msgr2=0x7f2c3410ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 shutdown_connections 2024-04-25T15:02:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.054+0000 7f2c3a600640 1 -- 172.21.15.19:0/4049551202 wait complete. 2024-04-25T15:02:01.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:01.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 -- 172.21.15.19:0/2515490838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08107ab0 msgr2=0x7f7b08109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 --2- 172.21.15.19:0/2515490838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08107ab0 0x7f7b08109f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009ca0 tx=0x7f7b0002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 -- 172.21.15.19:0/2515490838 shutdown_connections 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 --2- 172.21.15.19:0/2515490838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08107ab0 0x7f7b08109f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 --2- 172.21.15.19:0/2515490838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08105180 0x7f7b08107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 --2- 172.21.15.19:0/2515490838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08102850 0x7f7b08104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 -- 172.21.15.19:0/2515490838 >> 172.21.15.19:0/2515490838 conn(0x7f7b080fc220 msgr2=0x7f7b080fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 -- 172.21.15.19:0/2515490838 shutdown_connections 2024-04-25T15:02:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.271+0000 7f7b0e600640 1 -- 172.21.15.19:0/2515490838 wait complete. 2024-04-25T15:02:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.272+0000 7f7b0e600640 1 Processor -- start 2024-04-25T15:02:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.272+0000 7f7b0e600640 1 -- start start 2024-04-25T15:02:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08105180 0x7f7b08198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08107ab0 0x7f7b0819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810d920 con 0x7f7b08105180 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810d7a0 con 0x7f7b08107ab0 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810daa0 con 0x7f7b08102850 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b07e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08105180 0x7f7b08198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:33844/0 (socket says 172.21.15.19:33844) 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 -- 172.21.15.19:0/644844419 learned_addr learned my addr 172.21.15.19:0/644844419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0dc00640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08107ab0 0x7f7b0819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 -- 172.21.15.19:0/644844419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08107ab0 msgr2=0x7f7b0819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08107ab0 0x7f7b0819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 -- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08105180 msgr2=0x7f7b08198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08105180 0x7f7b08198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.273+0000 7f7b0d200640 1 -- 172.21.15.19:0/644844419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0819fc10 con 0x7f7b08102850 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b0d200640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d950 tx=0x7f7afc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017030 con 0x7f7b08102850 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0819ff00 con 0x7f7b08102850 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc004020 con 0x7f7b08102850 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0047a0 con 0x7f7b08102850 2024-04-25T15:02:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.274+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081a03c0 con 0x7f7b08102850 2024-04-25T15:02:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.275+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b080fdd10 con 0x7f7b08102850 2024-04-25T15:02:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.278+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7afc009a30 con 0x7f7b08102850 2024-04-25T15:02:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.279+0000 7f7b05600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 0x7f7af00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.280+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7afc0cfe10 con 0x7f7b08102850 2024-04-25T15:02:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.280+0000 7f7b07e00640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 0x7f7af00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.280+0000 7f7b07e00640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 0x7f7af00a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b08199500 tx=0x7f7af4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.285+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7afc0d3030 con 0x7f7b08102850 2024-04-25T15:02:01.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: pgmap v317: 33 pgs: 15 creating+peering, 1 active+clean, 17 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:01.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T15:02:01.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T15:02:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4049551202' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T15:02:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T15:02:01.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:01 smithi083 ceph-mon[32919]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: pgmap v317: 33 pgs: 15 creating+peering, 1 active+clean, 17 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4049551202' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T15:02:01.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:01 smithi102 ceph-mon[33005]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:02:01.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: pgmap v317: 33 pgs: 15 creating+peering, 1 active+clean, 17 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:01.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4049551202' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T15:02:01.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:01 smithi019 ceph-mon[28090]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:02:01.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.493+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b08062b80 con 0x7f7b08102850 2024-04-25T15:02:01.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.495+0000 7f7b05600640 1 -- 172.21.15.19:0/644844419 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22161 (secure 0 0 0) 0x7f7afc09c8a0 con 0x7f7b08102850 2024-04-25T15:02:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:01.497 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:00.518860+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:01.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 msgr2=0x7f7af00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 0x7f7af00a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b08199500 tx=0x7f7af4005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 msgr2=0x7f7b08197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d950 tx=0x7f7afc00de10 comp rx=0 tx=0).stop 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 shutdown_connections 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7af00a3bb0 0x7f7af00a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b08107ab0 0x7f7b0819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b08105180 0x7f7b08198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 --2- 172.21.15.19:0/644844419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b08102850 0x7f7b08197ff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 >> 172.21.15.19:0/644844419 conn(0x7f7b080fc220 msgr2=0x7f7b08103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 shutdown_connections 2024-04-25T15:02:01.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.498+0000 7f7b0e600640 1 -- 172.21.15.19:0/644844419 wait complete. 2024-04-25T15:02:01.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 -- 172.21.15.19:0/1774786343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68105340 msgr2=0x7f2b68107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 --2- 172.21.15.19:0/1774786343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68105340 0x7f2b68107730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009900 tx=0x7f2b5002a410 comp rx=0 tx=0).stop 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 -- 172.21.15.19:0/1774786343 shutdown_connections 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 --2- 172.21.15.19:0/1774786343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68107c70 0x7f2b6810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 --2- 172.21.15.19:0/1774786343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68105340 0x7f2b68107730 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 --2- 172.21.15.19:0/1774786343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68102a10 0x7f2b68104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.719+0000 7f2b6f400640 1 -- 172.21.15.19:0/1774786343 >> 172.21.15.19:0/1774786343 conn(0x7f2b680fc400 msgr2=0x7f2b680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:01.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.720+0000 7f2b6f400640 1 -- 172.21.15.19:0/1774786343 shutdown_connections 2024-04-25T15:02:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.720+0000 7f2b6f400640 1 -- 172.21.15.19:0/1774786343 wait complete. 2024-04-25T15:02:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 Processor -- start 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 -- start start 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68105340 0x7f2b6819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 0x7f2b681a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810dac0 con 0x7f2b68102a10 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810d940 con 0x7f2b68105340 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6f400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810dc40 con 0x7f2b68107c70 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44416/0 (socket says 172.21.15.19:44416) 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6ca00640 1 -- 172.21.15.19:0/468583532 learned_addr learned my addr 172.21.15.19:0/468583532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b6d400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 0x7f2b681a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.721+0000 7f2b67e00640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68105340 0x7f2b6819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 -- 172.21.15.19:0/468583532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68105340 msgr2=0x7f2b6819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68105340 0x7f2b6819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 -- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 msgr2=0x7f2b6819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 -- 172.21.15.19:0/468583532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681a2060 con 0x7f2b68107c70 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6ca00640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.722+0000 7f2b6d400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 0x7f2b681a19f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e970 tx=0x7f2b6000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.723+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000ccc0 con 0x7f2b68107c70 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.723+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b60005020 con 0x7f2b68107c70 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.723+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b681a2350 con 0x7f2b68107c70 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.723+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004a00 con 0x7f2b68107c70 2024-04-25T15:02:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.723+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681a2820 con 0x7f2b68107c70 2024-04-25T15:02:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.724+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b681097c0 con 0x7f2b68107c70 2024-04-25T15:02:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.726+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2b60004ba0 con 0x7f2b68107c70 2024-04-25T15:02:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.727+0000 7f2b65600640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 0x7f2b4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.728+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f2b600d02e0 con 0x7f2b68107c70 2024-04-25T15:02:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.728+0000 7f2b6ca00640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 0x7f2b4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.728+0000 7f2b6ca00640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 0x7f2b4c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b580049d0 tx=0x7f2b58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.732+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b6009cd70 con 0x7f2b68107c70 2024-04-25T15:02:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.942+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b68062b50 con 0x7f2b68107c70 2024-04-25T15:02:01.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.944+0000 7f2b65600640 1 -- 172.21.15.19:0/468583532 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22148 (secure 0 0 0) 0x7f2b6009c670 con 0x7f2b68107c70 2024-04-25T15:02:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:01.946 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:01.913953+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 msgr2=0x7f2b4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 0x7f2b4c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b580049d0 tx=0x7f2b58009500 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 msgr2=0x7f2b681a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 0x7f2b681a19f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e970 tx=0x7f2b6000ee30 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 shutdown_connections 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b68107c70 0x7f2b681a19f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2b4c0a3bb0 0x7f2b4c0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b68105340 0x7f2b6819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 --2- 172.21.15.19:0/468583532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b68102a10 0x7f2b6819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.946+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 >> 172.21.15.19:0/468583532 conn(0x7f2b680fc400 msgr2=0x7f2b681022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.947+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 shutdown_connections 2024-04-25T15:02:01.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:01.947+0000 7f2b6f400640 1 -- 172.21.15.19:0/468583532 wait complete. 2024-04-25T15:02:01.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 -- 172.21.15.19:0/4203284493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e010a160 msgr2=0x7fb4e01109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 --2- 172.21.15.19:0/4203284493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e010a160 0x7fb4e01109f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009ca0 tx=0x7fb4d802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 -- 172.21.15.19:0/4203284493 shutdown_connections 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 --2- 172.21.15.19:0/4203284493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e010a160 0x7fb4e01109f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 --2- 172.21.15.19:0/4203284493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 0x7fb4e0109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 --2- 172.21.15.19:0/4203284493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e01085b0 0x7fb4e01089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 -- 172.21.15.19:0/4203284493 >> 172.21.15.19:0/4203284493 conn(0x7fb4e0077550 msgr2=0x7fb4e0079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 -- 172.21.15.19:0/4203284493 shutdown_connections 2024-04-25T15:02:02.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.188+0000 7fb4e7800640 1 -- 172.21.15.19:0/4203284493 wait complete. 2024-04-25T15:02:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.189+0000 7fb4e7800640 1 Processor -- start 2024-04-25T15:02:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.189+0000 7fb4e7800640 1 -- start start 2024-04-25T15:02:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 0x7fb4e019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 0x7fb4e01a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0114110 con 0x7fb4e01085b0 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0113f90 con 0x7fb4e01097a0 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e7800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0114290 con 0x7fb4e010a160 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44428/0 (socket says 172.21.15.19:44428) 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 -- 172.21.15.19:0/2789566961 learned_addr learned my addr 172.21.15.19:0/2789566961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4dfe00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 0x7fb4e019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e5800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 0x7fb4e01a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 -- 172.21.15.19:0/2789566961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 msgr2=0x7fb4e01a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 0x7fb4e01a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 -- 172.21.15.19:0/2789566961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 msgr2=0x7fb4e019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 0x7fb4e019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e4e00640 1 -- 172.21.15.19:0/2789566961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e01a63a0 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.190+0000 7fb4e5800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 0x7fb4e01a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4e4e00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e9c0 tx=0x7fb4d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400cd80 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d40035f0 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01a6690 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4010690 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.191+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e01a6be0 con 0x7fb4e01085b0 2024-04-25T15:02:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.192+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e010f8d0 con 0x7fb4e01085b0 2024-04-25T15:02:02.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.196+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4d40037a0 con 0x7fb4e01085b0 2024-04-25T15:02:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.198+0000 7fb4dd600640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 0x7fb4c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.198+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb4d40d0720 con 0x7fb4e01085b0 2024-04-25T15:02:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.198+0000 7fb4dfe00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 0x7fb4c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:02.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.199+0000 7fb4dfe00640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 0x7fb4c80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb4e019fc90 tx=0x7fb4cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:02.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.203+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4d409d1f0 con 0x7fb4e01085b0 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/644844419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:02:02.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:02 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/468583532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:02.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:02.457+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb4e002d5c0 con 0x7fb4e01085b0 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/644844419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:02:02.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:02 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/468583532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/644844419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:02:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:02 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/468583532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.069+0000 7fb4dd600640 1 -- 172.21.15.19:0/2789566961 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fb4d409caf0 con 0x7fb4e01085b0 2024-04-25T15:02:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 msgr2=0x7fb4c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 0x7fb4c80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb4e019fc90 tx=0x7fb4cc005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 msgr2=0x7fb4e019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e9c0 tx=0x7fb4d400ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 shutdown_connections 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4e010a160 0x7fb4e01a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb4c80a3bb0 0x7fb4c80a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e01097a0 0x7fb4e019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 --2- 172.21.15.19:0/2789566961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4e01085b0 0x7fb4e019e780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 >> 172.21.15.19:0/2789566961 conn(0x7fb4e0077550 msgr2=0x7fb4e010caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 shutdown_connections 2024-04-25T15:02:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.073+0000 7fb4e7800640 1 -- 172.21.15.19:0/2789566961 wait complete. 2024-04-25T15:02:03.100 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T15:02:03.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 -- 172.21.15.19:0/235892130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 msgr2=0x7fc984101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 --2- 172.21.15.19:0/235892130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc984101ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009c90 tx=0x7fc97c02a880 comp rx=0 tx=0).stop 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 -- 172.21.15.19:0/235892130 shutdown_connections 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 --2- 172.21.15.19:0/235892130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc984101ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 --2- 172.21.15.19:0/235892130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 0x7fc984101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 --2- 172.21.15.19:0/235892130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 0x7fc984100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 -- 172.21.15.19:0/235892130 >> 172.21.15.19:0/235892130 conn(0x7fc9840fc300 msgr2=0x7fc9840fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:03.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 -- 172.21.15.19:0/235892130 shutdown_connections 2024-04-25T15:02:03.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.321+0000 7fc98a200640 1 -- 172.21.15.19:0/235892130 wait complete. 2024-04-25T15:02:03.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 Processor -- start 2024-04-25T15:02:03.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 -- start start 2024-04-25T15:02:03.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 0x7fc98419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 0x7fc98419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410fc10 con 0x7fc984101a40 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410fa90 con 0x7fc9841010a0 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.322+0000 7fc98a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410fd90 con 0x7fc984100740 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44440/0 (socket says 172.21.15.19:44440) 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 -- 172.21.15.19:0/2820608432 learned_addr learned my addr 172.21.15.19:0/2820608432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc988e00640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 0x7fc98419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc983e00640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 0x7fc98419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 -- 172.21.15.19:0/2820608432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 msgr2=0x7fc98419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 0x7fc98419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 -- 172.21.15.19:0/2820608432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 msgr2=0x7fc98419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 0x7fc98419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 -- 172.21.15.19:0/2820608432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a1e70 con 0x7fc984101a40 2024-04-25T15:02:03.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.323+0000 7fc989800640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c031900 tx=0x7fc97c004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:03.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.324+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c004e30 con 0x7fc984101a40 2024-04-25T15:02:03.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.324+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97c030a30 con 0x7fc984101a40 2024-04-25T15:02:03.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.324+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a2100 con 0x7fc984101a40 2024-04-25T15:02:03.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.324+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c005750 con 0x7fc984101a40 2024-04-25T15:02:03.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.324+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a26a0 con 0x7fc984101a40 2024-04-25T15:02:03.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.325+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98410b670 con 0x7fc984101a40 2024-04-25T15:02:03.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.327+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc97c0058f0 con 0x7fc984101a40 2024-04-25T15:02:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.329+0000 7fc981600640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 0x7fc96c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.329+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc97c0f14d0 con 0x7fc984101a40 2024-04-25T15:02:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.329+0000 7fc988e00640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 0x7fc96c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.330+0000 7fc988e00640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 0x7fc96c0a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc978009bd0 tx=0x7fc978005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:03.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.333+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc97c02e050 con 0x7fc984101a40 2024-04-25T15:02:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: pgmap v321: 97 pgs: 21 creating+peering, 13 active+clean, 63 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:active 2024-04-25T15:02:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T15:02:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T15:02:03.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:03 smithi019 ceph-mon[28090]: Cluster is now healthy 2024-04-25T15:02:03.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: pgmap v321: 97 pgs: 21 creating+peering, 13 active+clean, 63 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:active 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T15:02:03.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-mon[32919]: Cluster is now healthy 2024-04-25T15:02:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: pgmap v321: 97 pgs: 21 creating+peering, 13 active+clean, 63 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] up:active 2024-04-25T15:02:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:03.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T15:02:03.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T15:02:03.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:03 smithi102 ceph-mon[33005]: Cluster is now healthy 2024-04-25T15:02:03.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:03.582+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc984062b50 con 0x7fc984101a40 2024-04-25T15:02:04.040 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:02:03 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: 2024-04-25T15:02:03.742+0000 7fe7adc97700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T15:02:04.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.272+0000 7fc981600640 1 -- 172.21.15.19:0/2820608432 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fc97c0bdf60 con 0x7fc984101a40 2024-04-25T15:02:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 msgr2=0x7fc96c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 0x7fc96c0a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc978009bd0 tx=0x7fc978005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 msgr2=0x7fc9841a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc97c031900 tx=0x7fc97c004a90 comp rx=0 tx=0).stop 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 shutdown_connections 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc984101a40 0x7fc9841a1800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc96c0a3c30 0x7fc96c0a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9841010a0 0x7fc98419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 --2- 172.21.15.19:0/2820608432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc984100740 0x7fc98419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 >> 172.21.15.19:0/2820608432 conn(0x7fc9840fc300 msgr2=0x7fc9840fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 shutdown_connections 2024-04-25T15:02:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.277+0000 7fc98a200640 1 -- 172.21.15.19:0/2820608432 wait complete. 2024-04-25T15:02:04.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T15:02:04.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:04 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T15:02:04.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:04 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:04.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:04 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 -- 172.21.15.19:0/3447777091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501095a0 msgr2=0x7f9b50109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 --2- 172.21.15.19:0/3447777091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501095a0 0x7f9b50109a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009900 tx=0x7f9b3802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 -- 172.21.15.19:0/3447777091 shutdown_connections 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 --2- 172.21.15.19:0/3447777091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b50109f60 0x7f9b501107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 --2- 172.21.15.19:0/3447777091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501095a0 0x7f9b50109a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 --2- 172.21.15.19:0/3447777091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b501083b0 0x7f9b501087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 -- 172.21.15.19:0/3447777091 >> 172.21.15.19:0/3447777091 conn(0x7f9b50077350 msgr2=0x7f9b50079790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 -- 172.21.15.19:0/3447777091 shutdown_connections 2024-04-25T15:02:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.507+0000 7f9b54c00640 1 -- 172.21.15.19:0/3447777091 wait complete. 2024-04-25T15:02:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.508+0000 7f9b54c00640 1 Processor -- start 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.508+0000 7f9b54c00640 1 -- start start 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 0x7f9b5019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b50109f60 0x7f9b501a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50113ef0 con 0x7f9b501095a0 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50113d70 con 0x7f9b50109f60 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b54c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50114070 con 0x7f9b501083b0 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:40438/0 (socket says 172.21.15.19:40438) 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 -- 172.21.15.19:0/1314312059 learned_addr learned my addr 172.21.15.19:0/1314312059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b47e00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 0x7f9b5019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4fe00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b50109f60 0x7f9b501a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 -- 172.21.15.19:0/1314312059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b50109f60 msgr2=0x7f9b501a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b50109f60 0x7f9b501a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 -- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 msgr2=0x7f9b5019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 0x7f9b5019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b4f400640 1 -- 172.21.15.19:0/1314312059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b501a61f0 con 0x7f9b501083b0 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.509+0000 7f9b47e00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 0x7f9b5019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b4f400640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e9c0 tx=0x7f9b4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000cd80 con 0x7f9b501083b0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b501a64e0 con 0x7f9b501083b0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b400035f0 con 0x7f9b501083b0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40010690 con 0x7f9b501083b0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.510+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a69e0 con 0x7f9b501083b0 2024-04-25T15:02:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.511+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5010f6f0 con 0x7f9b501083b0 2024-04-25T15:02:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.514+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b40004000 con 0x7f9b501083b0 2024-04-25T15:02:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.515+0000 7f9b4cc00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 0x7f9b340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.515+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9b400d0a10 con 0x7f9b501083b0 2024-04-25T15:02:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.515+0000 7f9b47e00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 0x7f9b340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.516+0000 7f9b47e00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 0x7f9b340a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b5019fae0 tx=0x7f9b380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.518+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b4009d4a0 con 0x7f9b501083b0 2024-04-25T15:02:04.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:04 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T15:02:04.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:04 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:04.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:04 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T15:02:04.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:04 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2789566961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T15:02:04.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:04 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:04.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:04 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T15:02:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:04.770+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9b5002d5c0 con 0x7f9b501083b0 2024-04-25T15:02:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.291+0000 7f9b4cc00640 1 -- 172.21.15.19:0/1314312059 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f9b4009cda0 con 0x7f9b501083b0 2024-04-25T15:02:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.294+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 msgr2=0x7f9b340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.294+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 0x7f9b340a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b5019fae0 tx=0x7f9b380314a0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.294+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 msgr2=0x7f9b5019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.294+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e9c0 tx=0x7f9b4000ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 shutdown_connections 2024-04-25T15:02:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9b340a3bb0 0x7f9b340a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b50109f60 0x7f9b501a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b501095a0 0x7f9b5019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 --2- 172.21.15.19:0/1314312059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b501083b0 0x7f9b5019e670 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 >> 172.21.15.19:0/1314312059 conn(0x7f9b50077350 msgr2=0x7f9b5010c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 shutdown_connections 2024-04-25T15:02:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.295+0000 7f9b54c00640 1 -- 172.21.15.19:0/1314312059 wait complete. 2024-04-25T15:02:05.320 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T15:02:05.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 -- 172.21.15.19:0/226530278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4105280 msgr2=0x7fd7c4105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/226530278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4105280 0x7fd7c4105700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009900 tx=0x7fd7b002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 -- 172.21.15.19:0/226530278 shutdown_connections 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/226530278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105c40 0x7fd7c410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/226530278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4105280 0x7fd7c4105700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/226530278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4104090 0x7fd7c41044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 -- 172.21.15.19:0/226530278 >> 172.21.15.19:0/226530278 conn(0x7fd7c40ff600 msgr2=0x7fd7c4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 -- 172.21.15.19:0/226530278 shutdown_connections 2024-04-25T15:02:05.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.538+0000 7fd7c3e00640 1 -- 172.21.15.19:0/226530278 wait complete. 2024-04-25T15:02:05.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: pgmap v322: 97 pgs: 21 creating+peering, 54 active+clean, 22 unknown; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-04-25T15:02:05.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T15:02:05.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:05.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:05.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1314312059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:05 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.540+0000 7fd7c3e00640 1 Processor -- start 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.540+0000 7fd7c3e00640 1 -- start start 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.540+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.540+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 0x7fd7c419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 0x7fd7c41a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c410fc20 con 0x7fd7c4104090 2024-04-25T15:02:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c410faa0 con 0x7fd7c4105c40 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c410fda0 con 0x7fd7c4105280 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 0x7fd7c419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46046/0 (socket says 172.21.15.19:46046) 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2a00640 1 -- 172.21.15.19:0/1873145321 learned_addr learned my addr 172.21.15.19:0/1873145321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3400640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 0x7fd7c41a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 -- 172.21.15.19:0/1873145321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 msgr2=0x7fd7c41a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 0x7fd7c41a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 -- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 msgr2=0x7fd7c419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2000640 1 -- 172.21.15.19:0/1873145321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c41a1ed0 con 0x7fd7c4105280 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c3400640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 0x7fd7c41a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.541+0000 7fd7c2a00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7c2000640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 0x7fd7c419a7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b002a9e0 tx=0x7fd7b0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b003f070 con 0x7fd7c4105280 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c41a2160 con 0x7fd7c4105280 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b0031c50 con 0x7fd7c4105280 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0035030 con 0x7fd7c4105280 2024-04-25T15:02:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.542+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c41a2620 con 0x7fd7c4105280 2024-04-25T15:02:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.544+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c410b400 con 0x7fd7c4105280 2024-04-25T15:02:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.546+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd7b0042070 con 0x7fd7c4105280 2024-04-25T15:02:05.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.547+0000 7fd7bb400640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 0x7fd7a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:05.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.547+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd7b00f0a20 con 0x7fd7c4105280 2024-04-25T15:02:05.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.547+0000 7fd7c2a00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 0x7fd7a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:05.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.548+0000 7fd7c2a00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 0x7fd7a80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40095f0 tx=0x7fd7b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.551+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7b00bce60 con 0x7fd7c4105280 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: pgmap v322: 97 pgs: 21 creating+peering, 54 active+clean, 22 unknown; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1314312059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:05 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: pgmap v322: 97 pgs: 21 creating+peering, 54 active+clean, 22 unknown; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-04-25T15:02:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2820608432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T15:02:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T15:02:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1314312059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T15:02:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.809+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd7c4062390 con 0x7fd7a80a3bb0 2024-04-25T15:02:05.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.835+0000 7fd7bb400640 1 -- 172.21.15.19:0/1873145321 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd7c4062390 con 0x7fd7a80a3bb0 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 msgr2=0x7fd7a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 0x7fd7a80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40095f0 tx=0x7fd7b4005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 msgr2=0x7fd7c419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 0x7fd7c419a7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7b002a9e0 tx=0x7fd7b0003c10 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 shutdown_connections 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd7a80a3bb0 0x7fd7a80a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4105c40 0x7fd7c41a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c4105280 0x7fd7c419a7f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 --2- 172.21.15.19:0/1873145321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c4104090 0x7fd7c419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.837+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 >> 172.21.15.19:0/1873145321 conn(0x7fd7c40ff600 msgr2=0x7fd7c41085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.838+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 shutdown_connections 2024-04-25T15:02:05.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:05.838+0000 7fd7c3e00640 1 -- 172.21.15.19:0/1873145321 wait complete. 2024-04-25T15:02:05.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T15:02:06.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 -- 172.21.15.19:0/1616281229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0076910 msgr2=0x7fa9c0076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/1616281229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0076910 0x7fa9c0076d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009900 tx=0x7fa9b002a410 comp rx=0 tx=0).stop 2024-04-25T15:02:06.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 -- 172.21.15.19:0/1616281229 shutdown_connections 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/1616281229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 0x7fa9c0078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/1616281229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0077b50 0x7fa9c0077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/1616281229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0076910 0x7fa9c0076d30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 -- 172.21.15.19:0/1616281229 >> 172.21.15.19:0/1616281229 conn(0x7fa9c00ff600 msgr2=0x7fa9c0101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 -- 172.21.15.19:0/1616281229 shutdown_connections 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.049+0000 7fa9c4a00640 1 -- 172.21.15.19:0/1616281229 wait complete. 2024-04-25T15:02:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.051+0000 7fa9c4a00640 1 Processor -- start 2024-04-25T15:02:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.051+0000 7fa9c4a00640 1 -- start start 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.051+0000 7fa9c4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.051+0000 7fa9c4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0077b50 0x7fa9c019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.051+0000 7fa9c4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 0x7fa9c01a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9c4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0114010 con 0x7fa9c00784f0 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9c4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0113e90 con 0x7fa9c0077b50 2024-04-25T15:02:06.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9c4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0114190 con 0x7fa9c0076910 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:40494/0 (socket says 172.21.15.19:40494) 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 -- 172.21.15.19:0/612287617 learned_addr learned my addr 172.21.15.19:0/612287617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9bde00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0077b50 0x7fa9c019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9bf200640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 0x7fa9c01a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 -- 172.21.15.19:0/612287617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0077b50 msgr2=0x7fa9c019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0077b50 0x7fa9c019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 -- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 msgr2=0x7fa9c01a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 0x7fa9c01a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 -- 172.21.15.19:0/612287617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c01a62c0 con 0x7fa9c0076910 2024-04-25T15:02:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.052+0000 7fa9be800640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005ab0 tx=0x7fa9b0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.053+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b003f070 con 0x7fa9c0076910 2024-04-25T15:02:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.053+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b0031c50 con 0x7fa9c0076910 2024-04-25T15:02:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.053+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c01a6550 con 0x7fa9c0076910 2024-04-25T15:02:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.053+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0035030 con 0x7fa9c0076910 2024-04-25T15:02:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.053+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01a6a90 con 0x7fa9c0076910 2024-04-25T15:02:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.054+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c0110a00 con 0x7fa9c0076910 2024-04-25T15:02:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.056+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9b0042070 con 0x7fa9c0076910 2024-04-25T15:02:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.058+0000 7fa9b7400640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 0x7fa9a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.058+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa9b00f0a60 con 0x7fa9c0076910 2024-04-25T15:02:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.058+0000 7fa9bde00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 0x7fa9a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.059+0000 7fa9bde00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 0x7fa9a40a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0095f0 tx=0x7fa9ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.062+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b00bd4f0 con 0x7fa9c0076910 2024-04-25T15:02:06.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:02:05.818+0000 7f18da200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:06.122 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:02:05.818+0000 7f18da200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:06.122 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:02:05.818+0000 7f18da200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:06.122 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:02:05.818+0000 7f18da200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:06.122 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:02:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:02:05.818+0000 7f18da200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.300+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa9c0062420 con 0x7fa9a40a3c30 2024-04-25T15:02:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.318+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9b003b460 con 0x7fa9c0076910 2024-04-25T15:02:06.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.341+0000 7fa9b7400640 1 -- 172.21.15.19:0/612287617 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9c0062420 con 0x7fa9a40a3c30 2024-04-25T15:02:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 msgr2=0x7fa9a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 0x7fa9a40a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0095f0 tx=0x7fa9ac005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 msgr2=0x7fa9c019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005ab0 tx=0x7fa9b0003b90 comp rx=0 tx=0).stop 2024-04-25T15:02:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 shutdown_connections 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00784f0 0x7fa9c01a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa9a40a3c30 0x7fa9a40a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c0077b50 0x7fa9c019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 --2- 172.21.15.19:0/612287617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9c0076910 0x7fa9c019e6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 >> 172.21.15.19:0/612287617 conn(0x7fa9c00ff600 msgr2=0x7fa9c0102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 shutdown_connections 2024-04-25T15:02:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.344+0000 7fa9c4a00640 1 -- 172.21.15.19:0/612287617 wait complete. 2024-04-25T15:02:06.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T15:02:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:06 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T15:02:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:06 smithi102 ceph-mon[33005]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:06 smithi102 ceph-mon[33005]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.566+0000 7f4ee3e00640 1 -- 172.21.15.19:0/3962379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105c40 msgr2=0x7f4ee410c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.566+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/3962379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105c40 0x7f4ee410c4d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009be0 tx=0x7f4edc02a6d0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 -- 172.21.15.19:0/3962379350 shutdown_connections 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/3962379350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105c40 0x7f4ee410c4d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/3962379350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4105280 0x7f4ee4105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/3962379350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4104090 0x7f4ee41044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 -- 172.21.15.19:0/3962379350 >> 172.21.15.19:0/3962379350 conn(0x7f4ee40ff600 msgr2=0x7f4ee4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 -- 172.21.15.19:0/3962379350 shutdown_connections 2024-04-25T15:02:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.567+0000 7f4ee3e00640 1 -- 172.21.15.19:0/3962379350 wait complete. 2024-04-25T15:02:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.568+0000 7f4ee3e00640 1 Processor -- start 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.568+0000 7f4ee3e00640 1 -- start start 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.568+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 0x7f4ee419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4105c40 0x7f4ee41a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410fbd0 con 0x7f4ee4105280 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410fa50 con 0x7f4ee4105c40 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410fd50 con 0x7f4ee4104090 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46098/0 (socket says 172.21.15.19:46098) 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 -- 172.21.15.19:0/2992035385 learned_addr learned my addr 172.21.15.19:0/2992035385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2a00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 0x7f4ee419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee3400640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4105c40 0x7f4ee41a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 -- 172.21.15.19:0/2992035385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 msgr2=0x7f4ee419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 0x7f4ee419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 -- 172.21.15.19:0/2992035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4105c40 msgr2=0x7f4ee41a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4105c40 0x7f4ee41a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2000640 1 -- 172.21.15.19:0/2992035385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41a1ea0 con 0x7f4ee4105280 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.569+0000 7f4ee2a00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 0x7f4ee419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4ee2000640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e9f0 tx=0x7f4ed000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000cd80 con 0x7f4ee4105280 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ed00054d0 con 0x7f4ee4105280 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41a2190 con 0x7f4ee4105280 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0010690 con 0x7f4ee4105280 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.570+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a26e0 con 0x7f4ee4105280 2024-04-25T15:02:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.571+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee410b4d0 con 0x7f4ee4105280 2024-04-25T15:02:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.574+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ed0004e90 con 0x7f4ee4105280 2024-04-25T15:02:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.576+0000 7f4edb400640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 0x7f4ec80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:06.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.576+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4ed00d0870 con 0x7f4ee4105280 2024-04-25T15:02:06.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.576+0000 7f4ee2a00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 0x7f4ec80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:06.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.577+0000 7f4ee2a00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 0x7f4ec80a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40096c0 tx=0x7f4ed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:06.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.581+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ed009d1a0 con 0x7f4ee4105280 2024-04-25T15:02:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:06 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T15:02:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:06 smithi019 ceph-mon[28090]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:06 smithi019 ceph-mon[28090]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:06 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T15:02:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:06 smithi083 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T15:02:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:06 smithi083 ceph-mon[32919]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.809+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4ee4062390 con 0x7f4ec80a3cc0 2024-04-25T15:02:06.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.823+0000 7f4edb400640 1 -- 172.21.15.19:0/2992035385 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4ee4062390 con 0x7f4ec80a3cc0 2024-04-25T15:02:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/091ae1a4-fbee-4978-8a2a-3f9bf3981591 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 msgr2=0x7f4ec80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 0x7f4ec80a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40096c0 tx=0x7f4ed4009260 comp rx=0 tx=0).stop 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 msgr2=0x7f4ee419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e9f0 tx=0x7f4ed000edf0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 shutdown_connections 2024-04-25T15:02:06.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ec80a3cc0 0x7f4ec80a6180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4105c40 0x7f4ee41a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ee4105280 0x7f4ee419a810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 --2- 172.21.15.19:0/2992035385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ee4104090 0x7f4ee419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.825+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 >> 172.21.15.19:0/2992035385 conn(0x7f4ee40ff600 msgr2=0x7f4ee41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.826+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 shutdown_connections 2024-04-25T15:02:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:06.826+0000 7f4ee3e00640 1 -- 172.21.15.19:0/2992035385 wait complete. 2024-04-25T15:02:06.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T15:02:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 -- 172.21.15.19:0/664342413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 msgr2=0x7f53e81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 --2- 172.21.15.19:0/664342413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e81022b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009860 tx=0x7f53d802a340 comp rx=0 tx=0).stop 2024-04-25T15:02:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 -- 172.21.15.19:0/664342413 shutdown_connections 2024-04-25T15:02:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 --2- 172.21.15.19:0/664342413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 0x7f53e810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 --2- 172.21.15.19:0/664342413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 --2- 172.21.15.19:0/664342413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e81022b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 -- 172.21.15.19:0/664342413 >> 172.21.15.19:0/664342413 conn(0x7f53e80fd450 msgr2=0x7f53e80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 -- 172.21.15.19:0/664342413 shutdown_connections 2024-04-25T15:02:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.044+0000 7f53ee600640 1 -- 172.21.15.19:0/664342413 wait complete. 2024-04-25T15:02:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 Processor -- start 2024-04-25T15:02:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 -- start start 2024-04-25T15:02:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 0x7f53e81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810d9b0 con 0x7f53e8101e90 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810d830 con 0x7f53e8103a30 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.045+0000 7f53ee600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810db30 con 0x7f53e8103070 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46124/0 (socket says 172.21.15.19:46124) 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 -- 172.21.15.19:0/2576597358 learned_addr learned my addr 172.21.15.19:0/2576597358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53e7e00640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53edc00640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 0x7f53e81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 -- 172.21.15.19:0/2576597358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 msgr2=0x7f53e819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 -- 172.21.15.19:0/2576597358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 msgr2=0x7f53e81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 0x7f53e81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53ed200640 1 -- 172.21.15.19:0/2576597358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a1f20 con 0x7f53e8101e90 2024-04-25T15:02:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.046+0000 7f53e7e00640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53ed200640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53d80057c0 tx=0x7f53d8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8037040 con 0x7f53e8101e90 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d8030bd0 con 0x7f53e8101e90 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a21b0 con 0x7f53e8101e90 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8005280 con 0x7f53e8101e90 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.047+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a2640 con 0x7f53e8101e90 2024-04-25T15:02:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.048+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e81a2960 con 0x7f53e8101e90 2024-04-25T15:02:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.051+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53d8043020 con 0x7f53e8101e90 2024-04-25T15:02:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.052+0000 7f53e5600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 0x7f53cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.052+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f53d80f0310 con 0x7f53e8101e90 2024-04-25T15:02:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.052+0000 7f53e7e00640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 0x7f53cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.053+0000 7f53e7e00640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 0x7f53cc0a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53e819b810 tx=0x7f53d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.055+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53d804a020 con 0x7f53e8101e90 2024-04-25T15:02:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.293+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f53e80758a0 con 0x7f53cc0a3c40 2024-04-25T15:02:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.338+0000 7f53e5600640 1 -- 172.21.15.19:0/2576597358 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53e80758a0 con 0x7f53cc0a3c40 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.341+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 msgr2=0x7f53cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.341+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 0x7f53cc0a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53e819b810 tx=0x7f53d4009260 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.341+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 msgr2=0x7f53e819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.341+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53d80057c0 tx=0x7f53d8032da0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 shutdown_connections 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f53cc0a3c40 0x7f53cc0a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53e8103a30 0x7f53e81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e8103070 0x7f53e819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 --2- 172.21.15.19:0/2576597358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e8101e90 0x7f53e819a350 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 >> 172.21.15.19:0/2576597358 conn(0x7f53e80fd450 msgr2=0x7f53e8074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 shutdown_connections 2024-04-25T15:02:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.342+0000 7f53ee600640 1 -- 172.21.15.19:0/2576597358 wait complete. 2024-04-25T15:02:07.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T15:02:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.582+0000 7f1ec7e00640 1 -- 172.21.15.19:0/1871127606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec81006d0 msgr2=0x7f1ec8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.582+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/1871127606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec81006d0 0x7f1ec8100af0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009900 tx=0x7f1eb002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 -- 172.21.15.19:0/1871127606 shutdown_connections 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/1871127606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81019d0 0x7f1ec8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/1871127606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec8101030 0x7f1ec8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/1871127606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec81006d0 0x7f1ec8100af0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 -- 172.21.15.19:0/1871127606 >> 172.21.15.19:0/1871127606 conn(0x7f1ec80fc2f0 msgr2=0x7f1ec80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 -- 172.21.15.19:0/1871127606 shutdown_connections 2024-04-25T15:02:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.583+0000 7f1ec7e00640 1 -- 172.21.15.19:0/1871127606 wait complete. 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 Processor -- start 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 -- start start 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81006d0 0x7f1ec819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 0x7f1ec819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 0x7f1ec81a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810fbb0 con 0x7f1ec8101030 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810fa30 con 0x7f1ec81019d0 2024-04-25T15:02:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.584+0000 7f1ec7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810fd30 con 0x7f1ec81006d0 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 0x7f1ec819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 0x7f1ec81a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 0x7f1ec819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46144/0 (socket says 172.21.15.19:46144) 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec6000640 1 -- 172.21.15.19:0/3227146672 learned_addr learned my addr 172.21.15.19:0/3227146672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec6a00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81006d0 0x7f1ec819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 -- 172.21.15.19:0/3227146672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81006d0 msgr2=0x7f1ec819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81006d0 0x7f1ec819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 -- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 msgr2=0x7f1ec819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 0x7f1ec819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 -- 172.21.15.19:0/3227146672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a1e50 con 0x7f1ec81019d0 2024-04-25T15:02:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.585+0000 7f1ec7400640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 0x7f1ec81a17e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000e970 tx=0x7f1ec000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.586+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000ccc0 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.586+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec0005020 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.586+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a2140 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.586+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004a00 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.586+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a26e0 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.587+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec810b420 con 0x7f1ec81019d0 2024-04-25T15:02:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.589+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ec0004ba0 con 0x7f1ec81019d0 2024-04-25T15:02:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.590+0000 7f1ecca00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 0x7f1eac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.591+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1ec00d0500 con 0x7f1ec81019d0 2024-04-25T15:02:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.591+0000 7f1ec6a00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 0x7f1eac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.592+0000 7f1ec6a00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 0x7f1eac0a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:07.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.593+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ec009ce30 con 0x7f1ec81019d0 2024-04-25T15:02:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:07 smithi019 ceph-mon[28090]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T15:02:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:07 smithi019 ceph-mon[28090]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:07 smithi019 ceph-mon[28090]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T15:02:07.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:07 smithi019 ceph-mon[28090]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:07 smithi083 ceph-mon[32919]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T15:02:07.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:07 smithi083 ceph-mon[32919]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:07 smithi083 ceph-mon[32919]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T15:02:07.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:07 smithi083 ceph-mon[32919]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:07 smithi102 ceph-mon[33005]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T15:02:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:07 smithi102 ceph-mon[33005]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:07 smithi102 ceph-mon[33005]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T15:02:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:07 smithi102 ceph-mon[33005]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:07.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.834+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1ec8062390 con 0x7f1eac0a3cc0 2024-04-25T15:02:07.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.846+0000 7f1ecca00640 1 -- 172.21.15.19:0/3227146672 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1ec8062390 con 0x7f1eac0a3cc0 2024-04-25T15:02:07.848 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/94ee5623-1295-4128-a2e6-2e281e9fd96d 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 msgr2=0x7f1eac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 0x7f1eac0a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb00314a0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 msgr2=0x7f1ec81a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 0x7f1ec81a17e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000e970 tx=0x7f1ec000ee30 comp rx=0 tx=0).stop 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 shutdown_connections 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1eac0a3cc0 0x7f1eac0a6180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ec81019d0 0x7f1ec81a17e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec8101030 0x7f1ec819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 --2- 172.21.15.19:0/3227146672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ec81006d0 0x7f1ec819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 >> 172.21.15.19:0/3227146672 conn(0x7f1ec80fc2f0 msgr2=0x7f1ec80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 shutdown_connections 2024-04-25T15:02:07.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:07.849+0000 7f1ec7e00640 1 -- 172.21.15.19:0/3227146672 wait complete. 2024-04-25T15:02:07.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:02:08.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.072+0000 7ffa7aa00640 1 -- 172.21.15.19:0/2714693507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa740749b0 msgr2=0x7ffa74074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.072+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/2714693507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa740749b0 0x7ffa74074e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009900 tx=0x7ffa6002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 -- 172.21.15.19:0/2714693507 shutdown_connections 2024-04-25T15:02:08.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/2714693507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa74109cb0 0x7ffa7410c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/2714693507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa740749b0 0x7ffa74074e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/2714693507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74075f90 0x7ffa740743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 -- 172.21.15.19:0/2714693507 >> 172.21.15.19:0/2714693507 conn(0x7ffa740fd360 msgr2=0x7ffa740ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 -- 172.21.15.19:0/2714693507 shutdown_connections 2024-04-25T15:02:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.073+0000 7ffa7aa00640 1 -- 172.21.15.19:0/2714693507 wait complete. 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 Processor -- start 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 -- start start 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa74075f90 0x7ffa7419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 0x7ffa741a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7410fb70 con 0x7ffa740749b0 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7410f9f0 con 0x7ffa74075f90 2024-04-25T15:02:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa7aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7410fcf0 con 0x7ffa74109cb0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa79600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa79600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46158/0 (socket says 172.21.15.19:46158) 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.074+0000 7ffa79600640 1 -- 172.21.15.19:0/4284398255 learned_addr learned my addr 172.21.15.19:0/4284398255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa7a000640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 0x7ffa741a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa78c00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa74075f90 0x7ffa7419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 -- 172.21.15.19:0/4284398255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 msgr2=0x7ffa741a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 0x7ffa741a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 -- 172.21.15.19:0/4284398255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa74075f90 msgr2=0x7ffa7419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa74075f90 0x7ffa7419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 -- 172.21.15.19:0/4284398255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa741a1e60 con 0x7ffa740749b0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa7a000640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 0x7ffa741a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.075+0000 7ffa79600640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800eb90 tx=0x7ffa6800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.076+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68004d20 con 0x7ffa740749b0 2024-04-25T15:02:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.076+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa741a2150 con 0x7ffa740749b0 2024-04-25T15:02:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.076+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6800ee90 con 0x7ffa740749b0 2024-04-25T15:02:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.076+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68010670 con 0x7ffa740749b0 2024-04-25T15:02:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.076+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa741a2670 con 0x7ffa740749b0 2024-04-25T15:02:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.077+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa741089e0 con 0x7ffa740749b0 2024-04-25T15:02:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.079+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa68005020 con 0x7ffa740749b0 2024-04-25T15:02:08.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.080+0000 7ffa72000640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 0x7ffa5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.080+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ffa680d0b00 con 0x7ffa740749b0 2024-04-25T15:02:08.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.081+0000 7ffa78c00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 0x7ffa5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.081+0000 7ffa78c00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 0x7ffa5c0a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa60031500 tx=0x7ffa60005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:08.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.085+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa6809d430 con 0x7ffa740749b0 2024-04-25T15:02:08.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.333+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa7402d5c0 con 0x7ffa740749b0 2024-04-25T15:02:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.335+0000 7ffa72000640 1 -- 172.21.15.19:0/4284398255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7ffa7402d5c0 con 0x7ffa740749b0 2024-04-25T15:02:08.338 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:08.338 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:02:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 msgr2=0x7ffa5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 0x7ffa5c0a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa60031500 tx=0x7ffa60005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 msgr2=0x7ffa7419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800eb90 tx=0x7ffa6800c6a0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 shutdown_connections 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa74109cb0 0x7ffa741a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ffa5c0a3c40 0x7ffa5c0a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa74075f90 0x7ffa7419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 --2- 172.21.15.19:0/4284398255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa740749b0 0x7ffa7419a240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 >> 172.21.15.19:0/4284398255 conn(0x7ffa740fd360 msgr2=0x7ffa74105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:08.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.338+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 shutdown_connections 2024-04-25T15:02:08.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.339+0000 7ffa7aa00640 1 -- 172.21.15.19:0/4284398255 wait complete. 2024-04-25T15:02:08.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 -- 172.21.15.19:0/4142036124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de41083d0 msgr2=0x7f4de41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 --2- 172.21.15.19:0/4142036124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de41083d0 0x7f4de41087f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009ca0 tx=0x7f4dd402a8c0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 -- 172.21.15.19:0/4142036124 shutdown_connections 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 --2- 172.21.15.19:0/4142036124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de4109f80 0x7f4de4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 --2- 172.21.15.19:0/4142036124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41095c0 0x7f4de4109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 --2- 172.21.15.19:0/4142036124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de41083d0 0x7f4de41087f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 -- 172.21.15.19:0/4142036124 >> 172.21.15.19:0/4142036124 conn(0x7f4de4077390 msgr2=0x7f4de40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.572+0000 7f4de9200640 1 -- 172.21.15.19:0/4142036124 shutdown_connections 2024-04-25T15:02:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.573+0000 7f4de9200640 1 -- 172.21.15.19:0/4142036124 wait complete. 2024-04-25T15:02:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.573+0000 7f4de9200640 1 Processor -- start 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 -- start start 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41083d0 0x7f4de419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 0x7f4de41a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4113f60 con 0x7f4de41095c0 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4113de0 con 0x7f4de4109f80 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41140e0 con 0x7f4de41083d0 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46182/0 (socket says 172.21.15.19:46182) 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de2a00640 1 -- 172.21.15.19:0/3713534701 learned_addr learned my addr 172.21.15.19:0/3713534701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de3400640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41083d0 0x7f4de419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.574+0000 7f4de3e00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 0x7f4de41a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 -- 172.21.15.19:0/3713534701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41083d0 msgr2=0x7f4de419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41083d0 0x7f4de419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 -- 172.21.15.19:0/3713534701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 msgr2=0x7f4de41a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 0x7f4de41a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 -- 172.21.15.19:0/3713534701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41a6250 con 0x7f4de41095c0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de3e00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 0x7f4de41a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de2a00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000d770 tx=0x7f4dd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0017070 con 0x7f4de41095c0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de41a6540 con 0x7f4de41095c0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd00054d0 con 0x7f4de41095c0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.575+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0004c30 con 0x7f4de41095c0 2024-04-25T15:02:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.576+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41a6a90 con 0x7f4de41095c0 2024-04-25T15:02:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.577+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de410f6f0 con 0x7f4de41095c0 2024-04-25T15:02:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.579+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4dd0004e10 con 0x7f4de41095c0 2024-04-25T15:02:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.581+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 0x7f4dc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.581+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4dd00cffa0 con 0x7f4de41095c0 2024-04-25T15:02:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.581+0000 7f4de3400640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 0x7f4dc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.582+0000 7f4de3400640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 0x7f4dc80a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000f80 tx=0x7f4dd4005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.586+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dd009c8d0 con 0x7f4de41095c0 2024-04-25T15:02:08.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:08 smithi019 ceph-mon[28090]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:08.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:08 smithi083 ceph-mon[32919]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:08.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.796+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4de402d7c0 con 0x7f4de41095c0 2024-04-25T15:02:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.800+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3713534701 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22148 (secure 0 0 0) 0x7f4dd009c1d0 con 0x7f4de41095c0 2024-04-25T15:02:08.802 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:08.802 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:01.913953+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:08.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 msgr2=0x7f4dc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 0x7f4dc80a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000f80 tx=0x7f4dd4005f30 comp rx=0 tx=0).stop 2024-04-25T15:02:08.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 msgr2=0x7f4de419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000d770 tx=0x7f4dd000dc30 comp rx=0 tx=0).stop 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 shutdown_connections 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4dc80a3c40 0x7f4dc80a6100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4de4109f80 0x7f4de41a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41095c0 0x7f4de419eb70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 --2- 172.21.15.19:0/3713534701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4de41083d0 0x7f4de419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 >> 172.21.15.19:0/3713534701 conn(0x7f4de4077390 msgr2=0x7f4de410c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 shutdown_connections 2024-04-25T15:02:08.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:08.803+0000 7f4de9200640 1 -- 172.21.15.19:0/3713534701 wait complete. 2024-04-25T15:02:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:08 smithi102 ceph-mon[33005]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:08.824 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T15:02:08.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T15:02:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 -- 172.21.15.19:0/435700360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36901041d0 msgr2=0x7f36901045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 --2- 172.21.15.19:0/435700360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36901041d0 0x7f36901045f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3680009900 tx=0x7f368002a410 comp rx=0 tx=0).stop 2024-04-25T15:02:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 -- 172.21.15.19:0/435700360 shutdown_connections 2024-04-25T15:02:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 --2- 172.21.15.19:0/435700360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3690105d80 0x7f369010c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 --2- 172.21.15.19:0/435700360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 0x7f3690105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 --2- 172.21.15.19:0/435700360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36901041d0 0x7f36901045f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 -- 172.21.15.19:0/435700360 >> 172.21.15.19:0/435700360 conn(0x7f36900ff740 msgr2=0x7f3690101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 -- 172.21.15.19:0/435700360 shutdown_connections 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.034+0000 7f3696400640 1 -- 172.21.15.19:0/435700360 wait complete. 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.035+0000 7f3696400640 1 Processor -- start 2024-04-25T15:02:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.035+0000 7f3696400640 1 -- start start 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36901041d0 0x7f369019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 0x7f369019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f369010fd60 con 0x7f36901053c0 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f369010fbe0 con 0x7f3690105d80 2024-04-25T15:02:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f3696400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f369010fee0 con 0x7f36901041d0 2024-04-25T15:02:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f368f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36901041d0 0x7f369019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f368ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 0x7f369019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f368fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f368fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39838/0 (socket says 172.21.15.19:39838) 2024-04-25T15:02:09.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.036+0000 7f368fe00640 1 -- 172.21.15.19:0/2357602185 learned_addr learned my addr 172.21.15.19:0/2357602185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 -- 172.21.15.19:0/2357602185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36901041d0 msgr2=0x7f369019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36901041d0 0x7f369019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 -- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 msgr2=0x7f369019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 0x7f369019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 -- 172.21.15.19:0/2357602185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36901a2030 con 0x7f3690105d80 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.037+0000 7f368fe00640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f368800eb10 tx=0x7f368800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.038+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688004b20 con 0x7f3690105d80 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.038+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3688004cc0 con 0x7f3690105d80 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.038+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36901a2320 con 0x7f3690105d80 2024-04-25T15:02:09.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.038+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688010670 con 0x7f3690105d80 2024-04-25T15:02:09.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.038+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901a28c0 con 0x7f3690105d80 2024-04-25T15:02:09.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.039+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369010b570 con 0x7f3690105d80 2024-04-25T15:02:09.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.041+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3688005020 con 0x7f3690105d80 2024-04-25T15:02:09.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.042+0000 7f3687e00640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 0x7f36740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.043+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f36880d0930 con 0x7f3690105d80 2024-04-25T15:02:09.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.043+0000 7f368f400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 0x7f36740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.043+0000 7f368f400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 0x7f36740a6180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3680005ab0 tx=0x7f3680005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.046+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368809d260 con 0x7f3690105d80 2024-04-25T15:02:09.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.253+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3690062390 con 0x7f36740a3cc0 2024-04-25T15:02:09.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.265+0000 7f3687e00640 1 -- 172.21.15.19:0/2357602185 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258719 (secure 0 0 0) 0x7f3690062390 con 0x7f36740a3cc0 2024-04-25T15:02:09.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:09.272 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.269+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 msgr2=0x7f36740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.269+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 0x7f36740a6180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3680005ab0 tx=0x7f3680005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.269+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 msgr2=0x7f36901a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f368800eb10 tx=0x7f368800c2d0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 shutdown_connections 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f36740a3cc0 0x7f36740a6180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3690105d80 0x7f36901a19c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36901053c0 0x7f369019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 --2- 172.21.15.19:0/2357602185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36901041d0 0x7f369019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 >> 172.21.15.19:0/2357602185 conn(0x7f36900ff740 msgr2=0x7f3690108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 shutdown_connections 2024-04-25T15:02:09.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.270+0000 7f3696400640 1 -- 172.21.15.19:0/2357602185 wait complete. 2024-04-25T15:02:09.289 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T15:02:08.000629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612178,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":234,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374660,"kb_used_data":12948,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567164,"statfs":{"total":1151940427776,"available":1151556775936,"internally_reserved":0,"allocated":13258752,"data_stored":7230491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21810,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":13,"num_read_kb":16,"num_write":53,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.982245"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922333+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:02:01.922333+0000","last_peered":"2024-04-25T15:02:01.922333+0000","last_clean":"2024-04-25T15:02:01.922333+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:02:01.922333+0000","last_undegraded":"2024-04-25T15:02:01.922333+0000","last_fullsized":"2024-04-25T15:02:01.922333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196628+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:02:02.196628+0000","last_peered":"2024-04-25T15:02:02.196628+0000","last_clean":"2024-04-25T15:02:02.196628+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:02:02.196628+0000","last_undegraded":"2024-04-25T15:02:02.196628+0000","last_fullsized":"2024-04-25T15:02:02.196628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.197100+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:02:02.197100+0000","last_peered":"2024-04-25T15:02:02.197100+0000","last_clean":"2024-04-25T15:02:02.197100+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:02:02.197100+0000","last_undegraded":"2024-04-25T15:02:02.197100+0000","last_fullsized":"2024-04-25T15:02:02.197100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196053+0000","last_change":"2024-04-25T15:02:00.943534+0000","last_active":"2024-04-25T15:02:02.196053+0000","last_peered":"2024-04-25T15:02:02.196053+0000","last_clean":"2024-04-25T15:02:02.196053+0000","last_became_active":"2024-04-25T15:02:00.942018+0000","last_became_peered":"2024-04-25T15:02:00.942018+0000","last_unstale":"2024-04-25T15:02:02.196053+0000","last_undegraded":"2024-04-25T15:02:02.196053+0000","last_fullsized":"2024-04-25T15:02:02.196053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:54.003172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052197+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:02:02.052197+0000","last_peered":"2024-04-25T15:02:02.052197+0000","last_clean":"2024-04-25T15:02:02.052197+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:02:02.052197+0000","last_undegraded":"2024-04-25T15:02:02.052197+0000","last_fullsized":"2024-04-25T15:02:02.052197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919933+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:02:01.919933+0000","last_peered":"2024-04-25T15:02:01.919933+0000","last_clean":"2024-04-25T15:02:01.919933+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:02:01.919933+0000","last_undegraded":"2024-04-25T15:02:01.919933+0000","last_fullsized":"2024-04-25T15:02:01.919933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196562+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:02:02.196562+0000","last_peered":"2024-04-25T15:02:02.196562+0000","last_clean":"2024-04-25T15:02:02.196562+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:02:02.196562+0000","last_undegraded":"2024-04-25T15:02:02.196562+0000","last_fullsized":"2024-04-25T15:02:02.196562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053308+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:02:02.053308+0000","last_peered":"2024-04-25T15:02:02.053308+0000","last_clean":"2024-04-25T15:02:02.053308+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:02:02.053308+0000","last_undegraded":"2024-04-25T15:02:02.053308+0000","last_fullsized":"2024-04-25T15:02:02.053308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196484+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:02:02.196484+0000","last_peered":"2024-04-25T15:02:02.196484+0000","last_clean":"2024-04-25T15:02:02.196484+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:02:02.196484+0000","last_undegraded":"2024-04-25T15:02:02.196484+0000","last_fullsized":"2024-04-25T15:02:02.196484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.920304+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:02:01.920304+0000","last_peered":"2024-04-25T15:02:01.920304+0000","last_clean":"2024-04-25T15:02:01.920304+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:02:01.920304+0000","last_undegraded":"2024-04-25T15:02:01.920304+0000","last_fullsized":"2024-04-25T15:02:01.920304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196617+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:02:02.196617+0000","last_peered":"2024-04-25T15:02:02.196617+0000","last_clean":"2024-04-25T15:02:02.196617+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:02:02.196617+0000","last_undegraded":"2024-04-25T15:02:02.196617+0000","last_fullsized":"2024-04-25T15:02:02.196617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195955+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:02:02.195955+0000","last_peered":"2024-04-25T15:02:02.195955+0000","last_clean":"2024-04-25T15:02:02.195955+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:02:02.195955+0000","last_undegraded":"2024-04-25T15:02:02.195955+0000","last_fullsized":"2024-04-25T15:02:02.195955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.197059+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:02:02.197059+0000","last_peered":"2024-04-25T15:02:02.197059+0000","last_clean":"2024-04-25T15:02:02.197059+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:02:02.197059+0000","last_undegraded":"2024-04-25T15:02:02.197059+0000","last_fullsized":"2024-04-25T15:02:02.197059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195977+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:02:02.195977+0000","last_peered":"2024-04-25T15:02:02.195977+0000","last_clean":"2024-04-25T15:02:02.195977+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:02:02.195977+0000","last_undegraded":"2024-04-25T15:02:02.195977+0000","last_fullsized":"2024-04-25T15:02:02.195977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195708+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:02:02.195708+0000","last_peered":"2024-04-25T15:02:02.195708+0000","last_clean":"2024-04-25T15:02:02.195708+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:02:02.195708+0000","last_undegraded":"2024-04-25T15:02:02.195708+0000","last_fullsized":"2024-04-25T15:02:02.195708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195091+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:02:02.195091+0000","last_peered":"2024-04-25T15:02:02.195091+0000","last_clean":"2024-04-25T15:02:02.195091+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:02:02.195091+0000","last_undegraded":"2024-04-25T15:02:02.195091+0000","last_fullsized":"2024-04-25T15:02:02.195091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922386+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:02:01.922386+0000","last_peered":"2024-04-25T15:02:01.922386+0000","last_clean":"2024-04-25T15:02:01.922386+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:02:01.922386+0000","last_undegraded":"2024-04-25T15:02:01.922386+0000","last_fullsized":"2024-04-25T15:02:01.922386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921885+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:02:01.921885+0000","last_peered":"2024-04-25T15:02:01.921885+0000","last_clean":"2024-04-25T15:02:01.921885+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:02:01.921885+0000","last_undegraded":"2024-04-25T15:02:01.921885+0000","last_fullsized":"2024-04-25T15:02:01.921885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196067+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:02:02.196067+0000","last_peered":"2024-04-25T15:02:02.196067+0000","last_clean":"2024-04-25T15:02:02.196067+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:02:02.196067+0000","last_undegraded":"2024-04-25T15:02:02.196067+0000","last_fullsized":"2024-04-25T15:02:02.196067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921344+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:02:01.921344+0000","last_peered":"2024-04-25T15:02:01.921344+0000","last_clean":"2024-04-25T15:02:01.921344+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:02:01.921344+0000","last_undegraded":"2024-04-25T15:02:01.921344+0000","last_fullsized":"2024-04-25T15:02:01.921344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054095+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:02:02.054095+0000","last_peered":"2024-04-25T15:02:02.054095+0000","last_clean":"2024-04-25T15:02:02.054095+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:02:02.054095+0000","last_undegraded":"2024-04-25T15:02:02.054095+0000","last_fullsized":"2024-04-25T15:02:02.054095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054257+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:02:02.054257+0000","last_peered":"2024-04-25T15:02:02.054257+0000","last_clean":"2024-04-25T15:02:02.054257+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:02:02.054257+0000","last_undegraded":"2024-04-25T15:02:02.054257+0000","last_fullsized":"2024-04-25T15:02:02.054257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196586+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:02:02.196586+0000","last_peered":"2024-04-25T15:02:02.196586+0000","last_clean":"2024-04-25T15:02:02.196586+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:02:02.196586+0000","last_undegraded":"2024-04-25T15:02:02.196586+0000","last_fullsized":"2024-04-25T15:02:02.196586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052366+0000","last_change":"2024-04-25T15:02:00.041644+0000","last_active":"2024-04-25T15:02:02.052366+0000","last_peered":"2024-04-25T15:02:02.052366+0000","last_clean":"2024-04-25T15:02:02.052366+0000","last_became_active":"2024-04-25T15:02:00.041010+0000","last_became_peered":"2024-04-25T15:02:00.041010+0000","last_unstale":"2024-04-25T15:02:02.052366+0000","last_undegraded":"2024-04-25T15:02:02.052366+0000","last_fullsized":"2024-04-25T15:02:02.052366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:43.828081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196000+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:02:02.196000+0000","last_peered":"2024-04-25T15:02:02.196000+0000","last_clean":"2024-04-25T15:02:02.196000+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:02:02.196000+0000","last_undegraded":"2024-04-25T15:02:02.196000+0000","last_fullsized":"2024-04-25T15:02:02.196000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052277+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:02:02.052277+0000","last_peered":"2024-04-25T15:02:02.052277+0000","last_clean":"2024-04-25T15:02:02.052277+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:02:02.052277+0000","last_undegraded":"2024-04-25T15:02:02.052277+0000","last_fullsized":"2024-04-25T15:02:02.052277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196291+0000","last_change":"2024-04-25T15:02:00.945728+0000","last_active":"2024-04-25T15:02:02.196291+0000","last_peered":"2024-04-25T15:02:02.196291+0000","last_clean":"2024-04-25T15:02:02.196291+0000","last_became_active":"2024-04-25T15:02:00.945400+0000","last_became_peered":"2024-04-25T15:02:00.945400+0000","last_unstale":"2024-04-25T15:02:02.196291+0000","last_undegraded":"2024-04-25T15:02:02.196291+0000","last_fullsized":"2024-04-25T15:02:02.196291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:06.632919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195740+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:02:02.195740+0000","last_peered":"2024-04-25T15:02:02.195740+0000","last_clean":"2024-04-25T15:02:02.195740+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:02:02.195740+0000","last_undegraded":"2024-04-25T15:02:02.195740+0000","last_fullsized":"2024-04-25T15:02:02.195740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052427+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:02:02.052427+0000","last_peered":"2024-04-25T15:02:02.052427+0000","last_clean":"2024-04-25T15:02:02.052427+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:02:02.052427+0000","last_undegraded":"2024-04-25T15:02:02.052427+0000","last_fullsized":"2024-04-25T15:02:02.052427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919740+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:02:01.919740+0000","last_peered":"2024-04-25T15:02:01.919740+0000","last_clean":"2024-04-25T15:02:01.919740+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:02:01.919740+0000","last_undegraded":"2024-04-25T15:02:01.919740+0000","last_fullsized":"2024-04-25T15:02:01.919740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195912+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:02:02.195912+0000","last_peered":"2024-04-25T15:02:02.195912+0000","last_clean":"2024-04-25T15:02:02.195912+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:02:02.195912+0000","last_undegraded":"2024-04-25T15:02:02.195912+0000","last_fullsized":"2024-04-25T15:02:02.195912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196091+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:02:02.196091+0000","last_peered":"2024-04-25T15:02:02.196091+0000","last_clean":"2024-04-25T15:02:02.196091+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:02:02.196091+0000","last_undegraded":"2024-04-25T15:02:02.196091+0000","last_fullsized":"2024-04-25T15:02:02.196091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053453+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:02:02.053453+0000","last_peered":"2024-04-25T15:02:02.053453+0000","last_clean":"2024-04-25T15:02:02.053453+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:02:02.053453+0000","last_undegraded":"2024-04-25T15:02:02.053453+0000","last_fullsized":"2024-04-25T15:02:02.053453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195434+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:02:02.195434+0000","last_peered":"2024-04-25T15:02:02.195434+0000","last_clean":"2024-04-25T15:02:02.195434+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:02:02.195434+0000","last_undegraded":"2024-04-25T15:02:02.195434+0000","last_fullsized":"2024-04-25T15:02:02.195434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921233+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:02:01.921233+0000","last_peered":"2024-04-25T15:02:01.921233+0000","last_clean":"2024-04-25T15:02:01.921233+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:02:01.921233+0000","last_undegraded":"2024-04-25T15:02:01.921233+0000","last_fullsized":"2024-04-25T15:02:01.921233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.920093+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:02:01.920093+0000","last_peered":"2024-04-25T15:02:01.920093+0000","last_clean":"2024-04-25T15:02:01.920093+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:02:01.920093+0000","last_undegraded":"2024-04-25T15:02:01.920093+0000","last_fullsized":"2024-04-25T15:02:01.920093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196103+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:02:02.196103+0000","last_peered":"2024-04-25T15:02:02.196103+0000","last_clean":"2024-04-25T15:02:02.196103+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:02:02.196103+0000","last_undegraded":"2024-04-25T15:02:02.196103+0000","last_fullsized":"2024-04-25T15:02:02.196103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196621+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:02:02.196621+0000","last_peered":"2024-04-25T15:02:02.196621+0000","last_clean":"2024-04-25T15:02:02.196621+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:02:02.196621+0000","last_undegraded":"2024-04-25T15:02:02.196621+0000","last_fullsized":"2024-04-25T15:02:02.196621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196641+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:02:02.196641+0000","last_peered":"2024-04-25T15:02:02.196641+0000","last_clean":"2024-04-25T15:02:02.196641+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:02:02.196641+0000","last_undegraded":"2024-04-25T15:02:02.196641+0000","last_fullsized":"2024-04-25T15:02:02.196641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196329+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:02:02.196329+0000","last_peered":"2024-04-25T15:02:02.196329+0000","last_clean":"2024-04-25T15:02:02.196329+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:02:02.196329+0000","last_undegraded":"2024-04-25T15:02:02.196329+0000","last_fullsized":"2024-04-25T15:02:02.196329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196263+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:02:02.196263+0000","last_peered":"2024-04-25T15:02:02.196263+0000","last_clean":"2024-04-25T15:02:02.196263+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:02:02.196263+0000","last_undegraded":"2024-04-25T15:02:02.196263+0000","last_fullsized":"2024-04-25T15:02:02.196263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'13","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:06.328967+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:02:06.328967+0000","last_peered":"2024-04-25T15:02:06.328967+0000","last_clean":"2024-04-25T15:02:06.328967+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:02:06.328967+0000","last_undegraded":"2024-04-25T15:02:06.328967+0000","last_fullsized":"2024-04-25T15:02:06.328967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052321+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:02:02.052321+0000","last_peered":"2024-04-25T15:02:02.052321+0000","last_clean":"2024-04-25T15:02:02.052321+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:02:02.052321+0000","last_undegraded":"2024-04-25T15:02:02.052321+0000","last_fullsized":"2024-04-25T15:02:02.052321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195573+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:02:02.195573+0000","last_peered":"2024-04-25T15:02:02.195573+0000","last_clean":"2024-04-25T15:02:02.195573+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:02:02.195573+0000","last_undegraded":"2024-04-25T15:02:02.195573+0000","last_fullsized":"2024-04-25T15:02:02.195573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196868+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:02:02.196868+0000","last_peered":"2024-04-25T15:02:02.196868+0000","last_clean":"2024-04-25T15:02:02.196868+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:02:02.196868+0000","last_undegraded":"2024-04-25T15:02:02.196868+0000","last_fullsized":"2024-04-25T15:02:02.196868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195634+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:02:02.195634+0000","last_peered":"2024-04-25T15:02:02.195634+0000","last_clean":"2024-04-25T15:02:02.195634+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:02:02.195634+0000","last_undegraded":"2024-04-25T15:02:02.195634+0000","last_fullsized":"2024-04-25T15:02:02.195634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196056+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:02:02.196056+0000","last_peered":"2024-04-25T15:02:02.196056+0000","last_clean":"2024-04-25T15:02:02.196056+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:02:02.196056+0000","last_undegraded":"2024-04-25T15:02:02.196056+0000","last_fullsized":"2024-04-25T15:02:02.196056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195563+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:02:02.195563+0000","last_peered":"2024-04-25T15:02:02.195563+0000","last_clean":"2024-04-25T15:02:02.195563+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:02:02.195563+0000","last_undegraded":"2024-04-25T15:02:02.195563+0000","last_fullsized":"2024-04-25T15:02:02.195563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196130+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:02:02.196130+0000","last_peered":"2024-04-25T15:02:02.196130+0000","last_clean":"2024-04-25T15:02:02.196130+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:02:02.196130+0000","last_undegraded":"2024-04-25T15:02:02.196130+0000","last_fullsized":"2024-04-25T15:02:02.196130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.051871+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:02:02.051871+0000","last_peered":"2024-04-25T15:02:02.051871+0000","last_clean":"2024-04-25T15:02:02.051871+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:02:02.051871+0000","last_undegraded":"2024-04-25T15:02:02.051871+0000","last_fullsized":"2024-04-25T15:02:02.051871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195977+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:02:02.195977+0000","last_peered":"2024-04-25T15:02:02.195977+0000","last_clean":"2024-04-25T15:02:02.195977+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:02:02.195977+0000","last_undegraded":"2024-04-25T15:02:02.195977+0000","last_fullsized":"2024-04-25T15:02:02.195977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052479+0000","last_change":"2024-04-25T15:02:00.036587+0000","last_active":"2024-04-25T15:02:02.052479+0000","last_peered":"2024-04-25T15:02:02.052479+0000","last_clean":"2024-04-25T15:02:02.052479+0000","last_became_active":"2024-04-25T15:02:00.036225+0000","last_became_peered":"2024-04-25T15:02:00.036225+0000","last_unstale":"2024-04-25T15:02:02.052479+0000","last_undegraded":"2024-04-25T15:02:02.052479+0000","last_fullsized":"2024-04-25T15:02:02.052479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.381958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922286+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:02:01.922286+0000","last_peered":"2024-04-25T15:02:01.922286+0000","last_clean":"2024-04-25T15:02:01.922286+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:02:01.922286+0000","last_undegraded":"2024-04-25T15:02:01.922286+0000","last_fullsized":"2024-04-25T15:02:01.922286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921484+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:02:01.921484+0000","last_peered":"2024-04-25T15:02:01.921484+0000","last_clean":"2024-04-25T15:02:01.921484+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:02:01.921484+0000","last_undegraded":"2024-04-25T15:02:01.921484+0000","last_fullsized":"2024-04-25T15:02:01.921484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.246248+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:02:02.246248+0000","last_peered":"2024-04-25T15:02:02.246248+0000","last_clean":"2024-04-25T15:02:02.246248+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:02:02.246248+0000","last_undegraded":"2024-04-25T15:02:02.246248+0000","last_fullsized":"2024-04-25T15:02:02.246248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196250+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:02:02.196250+0000","last_peered":"2024-04-25T15:02:02.196250+0000","last_clean":"2024-04-25T15:02:02.196250+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:02:02.196250+0000","last_undegraded":"2024-04-25T15:02:02.196250+0000","last_fullsized":"2024-04-25T15:02:02.196250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919938+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:02:01.919938+0000","last_peered":"2024-04-25T15:02:01.919938+0000","last_clean":"2024-04-25T15:02:01.919938+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:02:01.919938+0000","last_undegraded":"2024-04-25T15:02:01.919938+0000","last_fullsized":"2024-04-25T15:02:01.919938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195665+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:02:02.195665+0000","last_peered":"2024-04-25T15:02:02.195665+0000","last_clean":"2024-04-25T15:02:02.195665+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:02:02.195665+0000","last_undegraded":"2024-04-25T15:02:02.195665+0000","last_fullsized":"2024-04-25T15:02:02.195665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195506+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:02:02.195506+0000","last_peered":"2024-04-25T15:02:02.195506+0000","last_clean":"2024-04-25T15:02:02.195506+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:02:02.195506+0000","last_undegraded":"2024-04-25T15:02:02.195506+0000","last_fullsized":"2024-04-25T15:02:02.195506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053661+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:02:02.053661+0000","last_peered":"2024-04-25T15:02:02.053661+0000","last_clean":"2024-04-25T15:02:02.053661+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:02:02.053661+0000","last_undegraded":"2024-04-25T15:02:02.053661+0000","last_fullsized":"2024-04-25T15:02:02.053661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195437+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:02:02.195437+0000","last_peered":"2024-04-25T15:02:02.195437+0000","last_clean":"2024-04-25T15:02:02.195437+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:02:02.195437+0000","last_undegraded":"2024-04-25T15:02:02.195437+0000","last_fullsized":"2024-04-25T15:02:02.195437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196312+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:02:02.196312+0000","last_peered":"2024-04-25T15:02:02.196312+0000","last_clean":"2024-04-25T15:02:02.196312+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:02:02.196312+0000","last_undegraded":"2024-04-25T15:02:02.196312+0000","last_fullsized":"2024-04-25T15:02:02.196312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195664+0000","last_change":"2024-04-25T15:02:00.945058+0000","last_active":"2024-04-25T15:02:02.195664+0000","last_peered":"2024-04-25T15:02:02.195664+0000","last_clean":"2024-04-25T15:02:02.195664+0000","last_became_active":"2024-04-25T15:02:00.944616+0000","last_became_peered":"2024-04-25T15:02:00.944616+0000","last_unstale":"2024-04-25T15:02:02.195664+0000","last_undegraded":"2024-04-25T15:02:02.195664+0000","last_fullsized":"2024-04-25T15:02:02.195664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:04.403170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196429+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:02:02.196429+0000","last_peered":"2024-04-25T15:02:02.196429+0000","last_clean":"2024-04-25T15:02:02.196429+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:02:02.196429+0000","last_undegraded":"2024-04-25T15:02:02.196429+0000","last_fullsized":"2024-04-25T15:02:02.196429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052134+0000","last_change":"2024-04-25T15:02:00.057832+0000","last_active":"2024-04-25T15:02:02.052134+0000","last_peered":"2024-04-25T15:02:02.052134+0000","last_clean":"2024-04-25T15:02:02.052134+0000","last_became_active":"2024-04-25T15:02:00.057536+0000","last_became_peered":"2024-04-25T15:02:00.057536+0000","last_unstale":"2024-04-25T15:02:02.052134+0000","last_undegraded":"2024-04-25T15:02:02.052134+0000","last_fullsized":"2024-04-25T15:02:02.052134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.453563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196978+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:02:02.196978+0000","last_peered":"2024-04-25T15:02:02.196978+0000","last_clean":"2024-04-25T15:02:02.196978+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:02:02.196978+0000","last_undegraded":"2024-04-25T15:02:02.196978+0000","last_fullsized":"2024-04-25T15:02:02.196978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922040+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:02:01.922040+0000","last_peered":"2024-04-25T15:02:01.922040+0000","last_clean":"2024-04-25T15:02:01.922040+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:02:01.922040+0000","last_undegraded":"2024-04-25T15:02:01.922040+0000","last_fullsized":"2024-04-25T15:02:01.922040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054434+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:02:02.054434+0000","last_peered":"2024-04-25T15:02:02.054434+0000","last_clean":"2024-04-25T15:02:02.054434+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:02:02.054434+0000","last_undegraded":"2024-04-25T15:02:02.054434+0000","last_fullsized":"2024-04-25T15:02:02.054434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195501+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:02:02.195501+0000","last_peered":"2024-04-25T15:02:02.195501+0000","last_clean":"2024-04-25T15:02:02.195501+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:02:02.195501+0000","last_undegraded":"2024-04-25T15:02:02.195501+0000","last_fullsized":"2024-04-25T15:02:02.195501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196811+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:02:02.196811+0000","last_peered":"2024-04-25T15:02:02.196811+0000","last_clean":"2024-04-25T15:02:02.196811+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:02:02.196811+0000","last_undegraded":"2024-04-25T15:02:02.196811+0000","last_fullsized":"2024-04-25T15:02:02.196811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195170+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:02:02.195170+0000","last_peered":"2024-04-25T15:02:02.195170+0000","last_clean":"2024-04-25T15:02:02.195170+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:02:02.195170+0000","last_undegraded":"2024-04-25T15:02:02.195170+0000","last_fullsized":"2024-04-25T15:02:02.195170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052141+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:02:02.052141+0000","last_peered":"2024-04-25T15:02:02.052141+0000","last_clean":"2024-04-25T15:02:02.052141+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:02:02.052141+0000","last_undegraded":"2024-04-25T15:02:02.052141+0000","last_fullsized":"2024-04-25T15:02:02.052141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053098+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:02:02.053098+0000","last_peered":"2024-04-25T15:02:02.053098+0000","last_clean":"2024-04-25T15:02:02.053098+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:02:02.053098+0000","last_undegraded":"2024-04-25T15:02:02.053098+0000","last_fullsized":"2024-04-25T15:02:02.053098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922086+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:02:01.922086+0000","last_peered":"2024-04-25T15:02:01.922086+0000","last_clean":"2024-04-25T15:02:01.922086+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:02:01.922086+0000","last_undegraded":"2024-04-25T15:02:01.922086+0000","last_fullsized":"2024-04-25T15:02:01.922086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195428+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:02:02.195428+0000","last_peered":"2024-04-25T15:02:02.195428+0000","last_clean":"2024-04-25T15:02:02.195428+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:02:02.195428+0000","last_undegraded":"2024-04-25T15:02:02.195428+0000","last_fullsized":"2024-04-25T15:02:02.195428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053905+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:02:02.053905+0000","last_peered":"2024-04-25T15:02:02.053905+0000","last_clean":"2024-04-25T15:02:02.053905+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:02:02.053905+0000","last_undegraded":"2024-04-25T15:02:02.053905+0000","last_fullsized":"2024-04-25T15:02:02.053905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052065+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:02:02.052065+0000","last_peered":"2024-04-25T15:02:02.052065+0000","last_clean":"2024-04-25T15:02:02.052065+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:02:02.052065+0000","last_undegraded":"2024-04-25T15:02:02.052065+0000","last_fullsized":"2024-04-25T15:02:02.052065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196430+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:02:02.196430+0000","last_peered":"2024-04-25T15:02:02.196430+0000","last_clean":"2024-04-25T15:02:02.196430+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:02:02.196430+0000","last_undegraded":"2024-04-25T15:02:02.196430+0000","last_fullsized":"2024-04-25T15:02:02.196430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195909+0000","last_change":"2024-04-25T15:02:00.060948+0000","last_active":"2024-04-25T15:02:02.195909+0000","last_peered":"2024-04-25T15:02:02.195909+0000","last_clean":"2024-04-25T15:02:02.195909+0000","last_became_active":"2024-04-25T15:02:00.060583+0000","last_became_peered":"2024-04-25T15:02:00.060583+0000","last_unstale":"2024-04-25T15:02:02.195909+0000","last_undegraded":"2024-04-25T15:02:02.195909+0000","last_fullsized":"2024-04-25T15:02:02.195909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:10.061940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196695+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:02:02.196695+0000","last_peered":"2024-04-25T15:02:02.196695+0000","last_clean":"2024-04-25T15:02:02.196695+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:02:02.196695+0000","last_undegraded":"2024-04-25T15:02:02.196695+0000","last_fullsized":"2024-04-25T15:02:02.196695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.955895+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:02:02.955895+0000","last_peered":"2024-04-25T15:02:02.955895+0000","last_clean":"2024-04-25T15:02:02.955895+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:02:02.955895+0000","last_undegraded":"2024-04-25T15:02:02.955895+0000","last_fullsized":"2024-04-25T15:02:02.955895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196223+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:02:02.196223+0000","last_peered":"2024-04-25T15:02:02.196223+0000","last_clean":"2024-04-25T15:02:02.196223+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:02:02.196223+0000","last_undegraded":"2024-04-25T15:02:02.196223+0000","last_fullsized":"2024-04-25T15:02:02.196223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196716+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:02:02.196716+0000","last_peered":"2024-04-25T15:02:02.196716+0000","last_clean":"2024-04-25T15:02:02.196716+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:02:02.196716+0000","last_undegraded":"2024-04-25T15:02:02.196716+0000","last_fullsized":"2024-04-25T15:02:02.196716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196884+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:02:02.196884+0000","last_peered":"2024-04-25T15:02:02.196884+0000","last_clean":"2024-04-25T15:02:02.196884+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:02:02.196884+0000","last_undegraded":"2024-04-25T15:02:02.196884+0000","last_fullsized":"2024-04-25T15:02:02.196884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922288+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:02:01.922288+0000","last_peered":"2024-04-25T15:02:01.922288+0000","last_clean":"2024-04-25T15:02:01.922288+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:02:01.922288+0000","last_undegraded":"2024-04-25T15:02:01.922288+0000","last_fullsized":"2024-04-25T15:02:01.922288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195675+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:02:02.195675+0000","last_peered":"2024-04-25T15:02:02.195675+0000","last_clean":"2024-04-25T15:02:02.195675+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:02:02.195675+0000","last_undegraded":"2024-04-25T15:02:02.195675+0000","last_fullsized":"2024-04-25T15:02:02.195675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195607+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:02:02.195607+0000","last_peered":"2024-04-25T15:02:02.195607+0000","last_clean":"2024-04-25T15:02:02.195607+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:02:02.195607+0000","last_undegraded":"2024-04-25T15:02:02.195607+0000","last_fullsized":"2024-04-25T15:02:02.195607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196311+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:02:02.196311+0000","last_peered":"2024-04-25T15:02:02.196311+0000","last_clean":"2024-04-25T15:02:02.196311+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:02:02.196311+0000","last_undegraded":"2024-04-25T15:02:02.196311+0000","last_fullsized":"2024-04-25T15:02:02.196311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196945+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:02:02.196945+0000","last_peered":"2024-04-25T15:02:02.196945+0000","last_clean":"2024-04-25T15:02:02.196945+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:02:02.196945+0000","last_undegraded":"2024-04-25T15:02:02.196945+0000","last_fullsized":"2024-04-25T15:02:02.196945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922265+0000","last_change":"2024-04-25T15:02:00.944416+0000","last_active":"2024-04-25T15:02:01.922265+0000","last_peered":"2024-04-25T15:02:01.922265+0000","last_clean":"2024-04-25T15:02:01.922265+0000","last_became_active":"2024-04-25T15:02:00.944025+0000","last_became_peered":"2024-04-25T15:02:00.944025+0000","last_unstale":"2024-04-25T15:02:01.922265+0000","last_undegraded":"2024-04-25T15:02:01.922265+0000","last_fullsized":"2024-04-25T15:02:01.922265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:53.828823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196216+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:02:02.196216+0000","last_peered":"2024-04-25T15:02:02.196216+0000","last_clean":"2024-04-25T15:02:02.196216+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:02:02.196216+0000","last_undegraded":"2024-04-25T15:02:02.196216+0000","last_fullsized":"2024-04-25T15:02:02.196216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054237+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:02:02.054237+0000","last_peered":"2024-04-25T15:02:02.054237+0000","last_clean":"2024-04-25T15:02:02.054237+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:02:02.054237+0000","last_undegraded":"2024-04-25T15:02:02.054237+0000","last_fullsized":"2024-04-25T15:02:02.054237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196120+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:02:02.196120+0000","last_peered":"2024-04-25T15:02:02.196120+0000","last_clean":"2024-04-25T15:02:02.196120+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:02:02.196120+0000","last_undegraded":"2024-04-25T15:02:02.196120+0000","last_fullsized":"2024-04-25T15:02:02.196120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053244+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:02:02.053244+0000","last_peered":"2024-04-25T15:02:02.053244+0000","last_clean":"2024-04-25T15:02:02.053244+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:02:02.053244+0000","last_undegraded":"2024-04-25T15:02:02.053244+0000","last_fullsized":"2024-04-25T15:02:02.053244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195675+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:02:02.195675+0000","last_peered":"2024-04-25T15:02:02.195675+0000","last_clean":"2024-04-25T15:02:02.195675+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:02:02.195675+0000","last_undegraded":"2024-04-25T15:02:02.195675+0000","last_fullsized":"2024-04-25T15:02:02.195675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196937+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:02:02.196937+0000","last_peered":"2024-04-25T15:02:02.196937+0000","last_clean":"2024-04-25T15:02:02.196937+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:02:02.196937+0000","last_undegraded":"2024-04-25T15:02:02.196937+0000","last_fullsized":"2024-04-25T15:02:02.196937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922061+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:02:01.922061+0000","last_peered":"2024-04-25T15:02:01.922061+0000","last_clean":"2024-04-25T15:02:01.922061+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:02:01.922061+0000","last_undegraded":"2024-04-25T15:02:01.922061+0000","last_fullsized":"2024-04-25T15:02:01.922061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":13,"num_read_kb":16,"num_write":52,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":33113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":452720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1064294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":453302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":453324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266727,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1544192,"data_stored":1043670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":452720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":454332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":452810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":454366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":452941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":452742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1552384,"data_stored":1043270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:02:09.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.535+0000 7fa2b3200640 1 -- 172.21.15.19:0/3180865405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b41051c0 msgr2=0x7fa2b4105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.535+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3180865405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b41051c0 0x7fa2b4105640 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009860 tx=0x7fa29c02a340 comp rx=0 tx=0).stop 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 -- 172.21.15.19:0/3180865405 shutdown_connections 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3180865405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b4105b80 0x7fa2b410c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3180865405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b41051c0 0x7fa2b4105640 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3180865405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 -- 172.21.15.19:0/3180865405 >> 172.21.15.19:0/3180865405 conn(0x7fa2b40ff560 msgr2=0x7fa2b4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 -- 172.21.15.19:0/3180865405 shutdown_connections 2024-04-25T15:02:09.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.536+0000 7fa2b3200640 1 -- 172.21.15.19:0/3180865405 wait complete. 2024-04-25T15:02:09.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.537+0000 7fa2b3200640 1 Processor -- start 2024-04-25T15:02:09.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.537+0000 7fa2b3200640 1 -- start start 2024-04-25T15:02:09.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b41051c0 0x7fa2b419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410fb10 con 0x7fa2b4105b80 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410f990 con 0x7fa2b4103fd0 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410fc90 con 0x7fa2b41051c0 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46214/0 (socket says 172.21.15.19:46214) 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 -- 172.21.15.19:0/3635366001 learned_addr learned my addr 172.21.15.19:0/3635366001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b1e00640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b1400640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b41051c0 0x7fa2b419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 -- 172.21.15.19:0/3635366001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b41051c0 msgr2=0x7fa2b419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b41051c0 0x7fa2b419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 -- 172.21.15.19:0/3635366001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 msgr2=0x7fa2b419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.538+0000 7fa2b2800640 1 -- 172.21.15.19:0/3635366001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b41a1e00 con 0x7fa2b4105b80 2024-04-25T15:02:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2b1e00640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2b2800640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e970 tx=0x7fa2ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac00ccc0 con 0x7fa2b4105b80 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac00ce60 con 0x7fa2b4105b80 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b41a20f0 con 0x7fa2b4105b80 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac010430 con 0x7fa2b4105b80 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.539+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b41a25c0 con 0x7fa2b4105b80 2024-04-25T15:02:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.541+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b410b340 con 0x7fa2b4105b80 2024-04-25T15:02:09.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.542+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa2ac004ff0 con 0x7fa2b4105b80 2024-04-25T15:02:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.544+0000 7fa2aaa00640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 0x7fa2980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.544+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa2ac0d08d0 con 0x7fa2b4105b80 2024-04-25T15:02:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.544+0000 7fa2b1e00640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 0x7fa2980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.545+0000 7fa2b1e00640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 0x7fa2980a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40098d0 tx=0x7fa2a4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:09.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.548+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2ac09d200 con 0x7fa2b4105b80 2024-04-25T15:02:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:09 smithi019 ceph-mon[28090]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:09 smithi019 ceph-mon[28090]: pgmap v324: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T15:02:09.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:09 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4284398255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:09 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3713534701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.708 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:09 smithi102 ceph-mon[33005]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:09.708 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:09 smithi102 ceph-mon[33005]: pgmap v324: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T15:02:09.709 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:09 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4284398255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.709 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:09 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3713534701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.756+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2b4062390 con 0x7fa2980a3c40 2024-04-25T15:02:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.774+0000 7fa2aaa00640 1 -- 172.21.15.19:0/3635366001 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258719 (secure 0 0 0) 0x7fa2b4062390 con 0x7fa2980a3c40 2024-04-25T15:02:09.777 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:09.780 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 msgr2=0x7fa2980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 0x7fa2980a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40098d0 tx=0x7fa2a4010040 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 msgr2=0x7fa2b41a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e970 tx=0x7fa2ac00ed70 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 shutdown_connections 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2b4105b80 0x7fa2b41a1790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b41051c0 0x7fa2b419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa2980a3c40 0x7fa2980a6100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 --2- 172.21.15.19:0/3635366001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b4103fd0 0x7fa2b419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 >> 172.21.15.19:0/3635366001 conn(0x7fa2b40ff560 msgr2=0x7fa2b4108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:09.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 shutdown_connections 2024-04-25T15:02:09.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:09.781+0000 7fa2b3200640 1 -- 172.21.15.19:0/3635366001 wait complete. 2024-04-25T15:02:09.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:09 smithi083 ceph-mon[32919]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:09.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:09 smithi083 ceph-mon[32919]: pgmap v324: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T15:02:09.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:09 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4284398255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:09 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3713534701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:09.802 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T15:02:08.000629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612178,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":234,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374660,"kb_used_data":12948,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567164,"statfs":{"total":1151940427776,"available":1151556775936,"internally_reserved":0,"allocated":13258752,"data_stored":7230491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21810,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":13,"num_read_kb":16,"num_write":53,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.982245"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922333+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:02:01.922333+0000","last_peered":"2024-04-25T15:02:01.922333+0000","last_clean":"2024-04-25T15:02:01.922333+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:02:01.922333+0000","last_undegraded":"2024-04-25T15:02:01.922333+0000","last_fullsized":"2024-04-25T15:02:01.922333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196628+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:02:02.196628+0000","last_peered":"2024-04-25T15:02:02.196628+0000","last_clean":"2024-04-25T15:02:02.196628+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:02:02.196628+0000","last_undegraded":"2024-04-25T15:02:02.196628+0000","last_fullsized":"2024-04-25T15:02:02.196628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.197100+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:02:02.197100+0000","last_peered":"2024-04-25T15:02:02.197100+0000","last_clean":"2024-04-25T15:02:02.197100+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:02:02.197100+0000","last_undegraded":"2024-04-25T15:02:02.197100+0000","last_fullsized":"2024-04-25T15:02:02.197100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196053+0000","last_change":"2024-04-25T15:02:00.943534+0000","last_active":"2024-04-25T15:02:02.196053+0000","last_peered":"2024-04-25T15:02:02.196053+0000","last_clean":"2024-04-25T15:02:02.196053+0000","last_became_active":"2024-04-25T15:02:00.942018+0000","last_became_peered":"2024-04-25T15:02:00.942018+0000","last_unstale":"2024-04-25T15:02:02.196053+0000","last_undegraded":"2024-04-25T15:02:02.196053+0000","last_fullsized":"2024-04-25T15:02:02.196053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:54.003172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052197+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:02:02.052197+0000","last_peered":"2024-04-25T15:02:02.052197+0000","last_clean":"2024-04-25T15:02:02.052197+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:02:02.052197+0000","last_undegraded":"2024-04-25T15:02:02.052197+0000","last_fullsized":"2024-04-25T15:02:02.052197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919933+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:02:01.919933+0000","last_peered":"2024-04-25T15:02:01.919933+0000","last_clean":"2024-04-25T15:02:01.919933+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:02:01.919933+0000","last_undegraded":"2024-04-25T15:02:01.919933+0000","last_fullsized":"2024-04-25T15:02:01.919933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196562+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:02:02.196562+0000","last_peered":"2024-04-25T15:02:02.196562+0000","last_clean":"2024-04-25T15:02:02.196562+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:02:02.196562+0000","last_undegraded":"2024-04-25T15:02:02.196562+0000","last_fullsized":"2024-04-25T15:02:02.196562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053308+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:02:02.053308+0000","last_peered":"2024-04-25T15:02:02.053308+0000","last_clean":"2024-04-25T15:02:02.053308+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:02:02.053308+0000","last_undegraded":"2024-04-25T15:02:02.053308+0000","last_fullsized":"2024-04-25T15:02:02.053308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196484+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:02:02.196484+0000","last_peered":"2024-04-25T15:02:02.196484+0000","last_clean":"2024-04-25T15:02:02.196484+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:02:02.196484+0000","last_undegraded":"2024-04-25T15:02:02.196484+0000","last_fullsized":"2024-04-25T15:02:02.196484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.920304+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:02:01.920304+0000","last_peered":"2024-04-25T15:02:01.920304+0000","last_clean":"2024-04-25T15:02:01.920304+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:02:01.920304+0000","last_undegraded":"2024-04-25T15:02:01.920304+0000","last_fullsized":"2024-04-25T15:02:01.920304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196617+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:02:02.196617+0000","last_peered":"2024-04-25T15:02:02.196617+0000","last_clean":"2024-04-25T15:02:02.196617+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:02:02.196617+0000","last_undegraded":"2024-04-25T15:02:02.196617+0000","last_fullsized":"2024-04-25T15:02:02.196617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195955+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:02:02.195955+0000","last_peered":"2024-04-25T15:02:02.195955+0000","last_clean":"2024-04-25T15:02:02.195955+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:02:02.195955+0000","last_undegraded":"2024-04-25T15:02:02.195955+0000","last_fullsized":"2024-04-25T15:02:02.195955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.197059+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:02:02.197059+0000","last_peered":"2024-04-25T15:02:02.197059+0000","last_clean":"2024-04-25T15:02:02.197059+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:02:02.197059+0000","last_undegraded":"2024-04-25T15:02:02.197059+0000","last_fullsized":"2024-04-25T15:02:02.197059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195977+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:02:02.195977+0000","last_peered":"2024-04-25T15:02:02.195977+0000","last_clean":"2024-04-25T15:02:02.195977+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:02:02.195977+0000","last_undegraded":"2024-04-25T15:02:02.195977+0000","last_fullsized":"2024-04-25T15:02:02.195977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195708+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:02:02.195708+0000","last_peered":"2024-04-25T15:02:02.195708+0000","last_clean":"2024-04-25T15:02:02.195708+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:02:02.195708+0000","last_undegraded":"2024-04-25T15:02:02.195708+0000","last_fullsized":"2024-04-25T15:02:02.195708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195091+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:02:02.195091+0000","last_peered":"2024-04-25T15:02:02.195091+0000","last_clean":"2024-04-25T15:02:02.195091+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:02:02.195091+0000","last_undegraded":"2024-04-25T15:02:02.195091+0000","last_fullsized":"2024-04-25T15:02:02.195091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922386+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:02:01.922386+0000","last_peered":"2024-04-25T15:02:01.922386+0000","last_clean":"2024-04-25T15:02:01.922386+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:02:01.922386+0000","last_undegraded":"2024-04-25T15:02:01.922386+0000","last_fullsized":"2024-04-25T15:02:01.922386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921885+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:02:01.921885+0000","last_peered":"2024-04-25T15:02:01.921885+0000","last_clean":"2024-04-25T15:02:01.921885+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:02:01.921885+0000","last_undegraded":"2024-04-25T15:02:01.921885+0000","last_fullsized":"2024-04-25T15:02:01.921885+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196067+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:02:02.196067+0000","last_peered":"2024-04-25T15:02:02.196067+0000","last_clean":"2024-04-25T15:02:02.196067+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:02:02.196067+0000","last_undegraded":"2024-04-25T15:02:02.196067+0000","last_fullsized":"2024-04-25T15:02:02.196067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921344+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:02:01.921344+0000","last_peered":"2024-04-25T15:02:01.921344+0000","last_clean":"2024-04-25T15:02:01.921344+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:02:01.921344+0000","last_undegraded":"2024-04-25T15:02:01.921344+0000","last_fullsized":"2024-04-25T15:02:01.921344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054095+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:02:02.054095+0000","last_peered":"2024-04-25T15:02:02.054095+0000","last_clean":"2024-04-25T15:02:02.054095+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:02:02.054095+0000","last_undegraded":"2024-04-25T15:02:02.054095+0000","last_fullsized":"2024-04-25T15:02:02.054095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054257+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:02:02.054257+0000","last_peered":"2024-04-25T15:02:02.054257+0000","last_clean":"2024-04-25T15:02:02.054257+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:02:02.054257+0000","last_undegraded":"2024-04-25T15:02:02.054257+0000","last_fullsized":"2024-04-25T15:02:02.054257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196586+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:02:02.196586+0000","last_peered":"2024-04-25T15:02:02.196586+0000","last_clean":"2024-04-25T15:02:02.196586+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:02:02.196586+0000","last_undegraded":"2024-04-25T15:02:02.196586+0000","last_fullsized":"2024-04-25T15:02:02.196586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052366+0000","last_change":"2024-04-25T15:02:00.041644+0000","last_active":"2024-04-25T15:02:02.052366+0000","last_peered":"2024-04-25T15:02:02.052366+0000","last_clean":"2024-04-25T15:02:02.052366+0000","last_became_active":"2024-04-25T15:02:00.041010+0000","last_became_peered":"2024-04-25T15:02:00.041010+0000","last_unstale":"2024-04-25T15:02:02.052366+0000","last_undegraded":"2024-04-25T15:02:02.052366+0000","last_fullsized":"2024-04-25T15:02:02.052366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:43.828081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196000+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:02:02.196000+0000","last_peered":"2024-04-25T15:02:02.196000+0000","last_clean":"2024-04-25T15:02:02.196000+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:02:02.196000+0000","last_undegraded":"2024-04-25T15:02:02.196000+0000","last_fullsized":"2024-04-25T15:02:02.196000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052277+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:02:02.052277+0000","last_peered":"2024-04-25T15:02:02.052277+0000","last_clean":"2024-04-25T15:02:02.052277+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:02:02.052277+0000","last_undegraded":"2024-04-25T15:02:02.052277+0000","last_fullsized":"2024-04-25T15:02:02.052277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196291+0000","last_change":"2024-04-25T15:02:00.945728+0000","last_active":"2024-04-25T15:02:02.196291+0000","last_peered":"2024-04-25T15:02:02.196291+0000","last_clean":"2024-04-25T15:02:02.196291+0000","last_became_active":"2024-04-25T15:02:00.945400+0000","last_became_peered":"2024-04-25T15:02:00.945400+0000","last_unstale":"2024-04-25T15:02:02.196291+0000","last_undegraded":"2024-04-25T15:02:02.196291+0000","last_fullsized":"2024-04-25T15:02:02.196291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:06.632919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195740+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:02:02.195740+0000","last_peered":"2024-04-25T15:02:02.195740+0000","last_clean":"2024-04-25T15:02:02.195740+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:02:02.195740+0000","last_undegraded":"2024-04-25T15:02:02.195740+0000","last_fullsized":"2024-04-25T15:02:02.195740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052427+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:02:02.052427+0000","last_peered":"2024-04-25T15:02:02.052427+0000","last_clean":"2024-04-25T15:02:02.052427+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:02:02.052427+0000","last_undegraded":"2024-04-25T15:02:02.052427+0000","last_fullsized":"2024-04-25T15:02:02.052427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919740+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:02:01.919740+0000","last_peered":"2024-04-25T15:02:01.919740+0000","last_clean":"2024-04-25T15:02:01.919740+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:02:01.919740+0000","last_undegraded":"2024-04-25T15:02:01.919740+0000","last_fullsized":"2024-04-25T15:02:01.919740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195912+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:02:02.195912+0000","last_peered":"2024-04-25T15:02:02.195912+0000","last_clean":"2024-04-25T15:02:02.195912+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:02:02.195912+0000","last_undegraded":"2024-04-25T15:02:02.195912+0000","last_fullsized":"2024-04-25T15:02:02.195912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196091+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:02:02.196091+0000","last_peered":"2024-04-25T15:02:02.196091+0000","last_clean":"2024-04-25T15:02:02.196091+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:02:02.196091+0000","last_undegraded":"2024-04-25T15:02:02.196091+0000","last_fullsized":"2024-04-25T15:02:02.196091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053453+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:02:02.053453+0000","last_peered":"2024-04-25T15:02:02.053453+0000","last_clean":"2024-04-25T15:02:02.053453+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:02:02.053453+0000","last_undegraded":"2024-04-25T15:02:02.053453+0000","last_fullsized":"2024-04-25T15:02:02.053453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195434+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:02:02.195434+0000","last_peered":"2024-04-25T15:02:02.195434+0000","last_clean":"2024-04-25T15:02:02.195434+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:02:02.195434+0000","last_undegraded":"2024-04-25T15:02:02.195434+0000","last_fullsized":"2024-04-25T15:02:02.195434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921233+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:02:01.921233+0000","last_peered":"2024-04-25T15:02:01.921233+0000","last_clean":"2024-04-25T15:02:01.921233+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:02:01.921233+0000","last_undegraded":"2024-04-25T15:02:01.921233+0000","last_fullsized":"2024-04-25T15:02:01.921233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.920093+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:02:01.920093+0000","last_peered":"2024-04-25T15:02:01.920093+0000","last_clean":"2024-04-25T15:02:01.920093+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:02:01.920093+0000","last_undegraded":"2024-04-25T15:02:01.920093+0000","last_fullsized":"2024-04-25T15:02:01.920093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196103+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:02:02.196103+0000","last_peered":"2024-04-25T15:02:02.196103+0000","last_clean":"2024-04-25T15:02:02.196103+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:02:02.196103+0000","last_undegraded":"2024-04-25T15:02:02.196103+0000","last_fullsized":"2024-04-25T15:02:02.196103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196621+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:02:02.196621+0000","last_peered":"2024-04-25T15:02:02.196621+0000","last_clean":"2024-04-25T15:02:02.196621+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:02:02.196621+0000","last_undegraded":"2024-04-25T15:02:02.196621+0000","last_fullsized":"2024-04-25T15:02:02.196621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196641+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:02:02.196641+0000","last_peered":"2024-04-25T15:02:02.196641+0000","last_clean":"2024-04-25T15:02:02.196641+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:02:02.196641+0000","last_undegraded":"2024-04-25T15:02:02.196641+0000","last_fullsized":"2024-04-25T15:02:02.196641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196329+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:02:02.196329+0000","last_peered":"2024-04-25T15:02:02.196329+0000","last_clean":"2024-04-25T15:02:02.196329+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:02:02.196329+0000","last_undegraded":"2024-04-25T15:02:02.196329+0000","last_fullsized":"2024-04-25T15:02:02.196329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196263+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:02:02.196263+0000","last_peered":"2024-04-25T15:02:02.196263+0000","last_clean":"2024-04-25T15:02:02.196263+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:02:02.196263+0000","last_undegraded":"2024-04-25T15:02:02.196263+0000","last_fullsized":"2024-04-25T15:02:02.196263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'13","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:06.328967+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:02:06.328967+0000","last_peered":"2024-04-25T15:02:06.328967+0000","last_clean":"2024-04-25T15:02:06.328967+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:02:06.328967+0000","last_undegraded":"2024-04-25T15:02:06.328967+0000","last_fullsized":"2024-04-25T15:02:06.328967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052321+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:02:02.052321+0000","last_peered":"2024-04-25T15:02:02.052321+0000","last_clean":"2024-04-25T15:02:02.052321+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:02:02.052321+0000","last_undegraded":"2024-04-25T15:02:02.052321+0000","last_fullsized":"2024-04-25T15:02:02.052321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195573+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:02:02.195573+0000","last_peered":"2024-04-25T15:02:02.195573+0000","last_clean":"2024-04-25T15:02:02.195573+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:02:02.195573+0000","last_undegraded":"2024-04-25T15:02:02.195573+0000","last_fullsized":"2024-04-25T15:02:02.195573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196868+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:02:02.196868+0000","last_peered":"2024-04-25T15:02:02.196868+0000","last_clean":"2024-04-25T15:02:02.196868+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:02:02.196868+0000","last_undegraded":"2024-04-25T15:02:02.196868+0000","last_fullsized":"2024-04-25T15:02:02.196868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195634+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:02:02.195634+0000","last_peered":"2024-04-25T15:02:02.195634+0000","last_clean":"2024-04-25T15:02:02.195634+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:02:02.195634+0000","last_undegraded":"2024-04-25T15:02:02.195634+0000","last_fullsized":"2024-04-25T15:02:02.195634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196056+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:02:02.196056+0000","last_peered":"2024-04-25T15:02:02.196056+0000","last_clean":"2024-04-25T15:02:02.196056+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:02:02.196056+0000","last_undegraded":"2024-04-25T15:02:02.196056+0000","last_fullsized":"2024-04-25T15:02:02.196056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195563+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:02:02.195563+0000","last_peered":"2024-04-25T15:02:02.195563+0000","last_clean":"2024-04-25T15:02:02.195563+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:02:02.195563+0000","last_undegraded":"2024-04-25T15:02:02.195563+0000","last_fullsized":"2024-04-25T15:02:02.195563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196130+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:02:02.196130+0000","last_peered":"2024-04-25T15:02:02.196130+0000","last_clean":"2024-04-25T15:02:02.196130+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:02:02.196130+0000","last_undegraded":"2024-04-25T15:02:02.196130+0000","last_fullsized":"2024-04-25T15:02:02.196130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.051871+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:02:02.051871+0000","last_peered":"2024-04-25T15:02:02.051871+0000","last_clean":"2024-04-25T15:02:02.051871+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:02:02.051871+0000","last_undegraded":"2024-04-25T15:02:02.051871+0000","last_fullsized":"2024-04-25T15:02:02.051871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195977+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:02:02.195977+0000","last_peered":"2024-04-25T15:02:02.195977+0000","last_clean":"2024-04-25T15:02:02.195977+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:02:02.195977+0000","last_undegraded":"2024-04-25T15:02:02.195977+0000","last_fullsized":"2024-04-25T15:02:02.195977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052479+0000","last_change":"2024-04-25T15:02:00.036587+0000","last_active":"2024-04-25T15:02:02.052479+0000","last_peered":"2024-04-25T15:02:02.052479+0000","last_clean":"2024-04-25T15:02:02.052479+0000","last_became_active":"2024-04-25T15:02:00.036225+0000","last_became_peered":"2024-04-25T15:02:00.036225+0000","last_unstale":"2024-04-25T15:02:02.052479+0000","last_undegraded":"2024-04-25T15:02:02.052479+0000","last_fullsized":"2024-04-25T15:02:02.052479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.381958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922286+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:02:01.922286+0000","last_peered":"2024-04-25T15:02:01.922286+0000","last_clean":"2024-04-25T15:02:01.922286+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:02:01.922286+0000","last_undegraded":"2024-04-25T15:02:01.922286+0000","last_fullsized":"2024-04-25T15:02:01.922286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.921484+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:02:01.921484+0000","last_peered":"2024-04-25T15:02:01.921484+0000","last_clean":"2024-04-25T15:02:01.921484+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:02:01.921484+0000","last_undegraded":"2024-04-25T15:02:01.921484+0000","last_fullsized":"2024-04-25T15:02:01.921484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.246248+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:02:02.246248+0000","last_peered":"2024-04-25T15:02:02.246248+0000","last_clean":"2024-04-25T15:02:02.246248+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:02:02.246248+0000","last_undegraded":"2024-04-25T15:02:02.246248+0000","last_fullsized":"2024-04-25T15:02:02.246248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196250+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:02:02.196250+0000","last_peered":"2024-04-25T15:02:02.196250+0000","last_clean":"2024-04-25T15:02:02.196250+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:02:02.196250+0000","last_undegraded":"2024-04-25T15:02:02.196250+0000","last_fullsized":"2024-04-25T15:02:02.196250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.919938+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:02:01.919938+0000","last_peered":"2024-04-25T15:02:01.919938+0000","last_clean":"2024-04-25T15:02:01.919938+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:02:01.919938+0000","last_undegraded":"2024-04-25T15:02:01.919938+0000","last_fullsized":"2024-04-25T15:02:01.919938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195665+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:02:02.195665+0000","last_peered":"2024-04-25T15:02:02.195665+0000","last_clean":"2024-04-25T15:02:02.195665+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:02:02.195665+0000","last_undegraded":"2024-04-25T15:02:02.195665+0000","last_fullsized":"2024-04-25T15:02:02.195665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195506+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:02:02.195506+0000","last_peered":"2024-04-25T15:02:02.195506+0000","last_clean":"2024-04-25T15:02:02.195506+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:02:02.195506+0000","last_undegraded":"2024-04-25T15:02:02.195506+0000","last_fullsized":"2024-04-25T15:02:02.195506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053661+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:02:02.053661+0000","last_peered":"2024-04-25T15:02:02.053661+0000","last_clean":"2024-04-25T15:02:02.053661+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:02:02.053661+0000","last_undegraded":"2024-04-25T15:02:02.053661+0000","last_fullsized":"2024-04-25T15:02:02.053661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195437+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:02:02.195437+0000","last_peered":"2024-04-25T15:02:02.195437+0000","last_clean":"2024-04-25T15:02:02.195437+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:02:02.195437+0000","last_undegraded":"2024-04-25T15:02:02.195437+0000","last_fullsized":"2024-04-25T15:02:02.195437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196312+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:02:02.196312+0000","last_peered":"2024-04-25T15:02:02.196312+0000","last_clean":"2024-04-25T15:02:02.196312+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:02:02.196312+0000","last_undegraded":"2024-04-25T15:02:02.196312+0000","last_fullsized":"2024-04-25T15:02:02.196312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195664+0000","last_change":"2024-04-25T15:02:00.945058+0000","last_active":"2024-04-25T15:02:02.195664+0000","last_peered":"2024-04-25T15:02:02.195664+0000","last_clean":"2024-04-25T15:02:02.195664+0000","last_became_active":"2024-04-25T15:02:00.944616+0000","last_became_peered":"2024-04-25T15:02:00.944616+0000","last_unstale":"2024-04-25T15:02:02.195664+0000","last_undegraded":"2024-04-25T15:02:02.195664+0000","last_fullsized":"2024-04-25T15:02:02.195664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:04.403170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196429+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:02:02.196429+0000","last_peered":"2024-04-25T15:02:02.196429+0000","last_clean":"2024-04-25T15:02:02.196429+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:02:02.196429+0000","last_undegraded":"2024-04-25T15:02:02.196429+0000","last_fullsized":"2024-04-25T15:02:02.196429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052134+0000","last_change":"2024-04-25T15:02:00.057832+0000","last_active":"2024-04-25T15:02:02.052134+0000","last_peered":"2024-04-25T15:02:02.052134+0000","last_clean":"2024-04-25T15:02:02.052134+0000","last_became_active":"2024-04-25T15:02:00.057536+0000","last_became_peered":"2024-04-25T15:02:00.057536+0000","last_unstale":"2024-04-25T15:02:02.052134+0000","last_undegraded":"2024-04-25T15:02:02.052134+0000","last_fullsized":"2024-04-25T15:02:02.052134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.453563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196978+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:02:02.196978+0000","last_peered":"2024-04-25T15:02:02.196978+0000","last_clean":"2024-04-25T15:02:02.196978+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:02:02.196978+0000","last_undegraded":"2024-04-25T15:02:02.196978+0000","last_fullsized":"2024-04-25T15:02:02.196978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922040+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:02:01.922040+0000","last_peered":"2024-04-25T15:02:01.922040+0000","last_clean":"2024-04-25T15:02:01.922040+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:02:01.922040+0000","last_undegraded":"2024-04-25T15:02:01.922040+0000","last_fullsized":"2024-04-25T15:02:01.922040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054434+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:02:02.054434+0000","last_peered":"2024-04-25T15:02:02.054434+0000","last_clean":"2024-04-25T15:02:02.054434+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:02:02.054434+0000","last_undegraded":"2024-04-25T15:02:02.054434+0000","last_fullsized":"2024-04-25T15:02:02.054434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195501+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:02:02.195501+0000","last_peered":"2024-04-25T15:02:02.195501+0000","last_clean":"2024-04-25T15:02:02.195501+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:02:02.195501+0000","last_undegraded":"2024-04-25T15:02:02.195501+0000","last_fullsized":"2024-04-25T15:02:02.195501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196811+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:02:02.196811+0000","last_peered":"2024-04-25T15:02:02.196811+0000","last_clean":"2024-04-25T15:02:02.196811+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:02:02.196811+0000","last_undegraded":"2024-04-25T15:02:02.196811+0000","last_fullsized":"2024-04-25T15:02:02.196811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195170+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:02:02.195170+0000","last_peered":"2024-04-25T15:02:02.195170+0000","last_clean":"2024-04-25T15:02:02.195170+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:02:02.195170+0000","last_undegraded":"2024-04-25T15:02:02.195170+0000","last_fullsized":"2024-04-25T15:02:02.195170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052141+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:02:02.052141+0000","last_peered":"2024-04-25T15:02:02.052141+0000","last_clean":"2024-04-25T15:02:02.052141+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:02:02.052141+0000","last_undegraded":"2024-04-25T15:02:02.052141+0000","last_fullsized":"2024-04-25T15:02:02.052141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053098+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:02:02.053098+0000","last_peered":"2024-04-25T15:02:02.053098+0000","last_clean":"2024-04-25T15:02:02.053098+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:02:02.053098+0000","last_undegraded":"2024-04-25T15:02:02.053098+0000","last_fullsized":"2024-04-25T15:02:02.053098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922086+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:02:01.922086+0000","last_peered":"2024-04-25T15:02:01.922086+0000","last_clean":"2024-04-25T15:02:01.922086+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:02:01.922086+0000","last_undegraded":"2024-04-25T15:02:01.922086+0000","last_fullsized":"2024-04-25T15:02:01.922086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195428+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:02:02.195428+0000","last_peered":"2024-04-25T15:02:02.195428+0000","last_clean":"2024-04-25T15:02:02.195428+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:02:02.195428+0000","last_undegraded":"2024-04-25T15:02:02.195428+0000","last_fullsized":"2024-04-25T15:02:02.195428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053905+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:02:02.053905+0000","last_peered":"2024-04-25T15:02:02.053905+0000","last_clean":"2024-04-25T15:02:02.053905+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:02:02.053905+0000","last_undegraded":"2024-04-25T15:02:02.053905+0000","last_fullsized":"2024-04-25T15:02:02.053905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.052065+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:02:02.052065+0000","last_peered":"2024-04-25T15:02:02.052065+0000","last_clean":"2024-04-25T15:02:02.052065+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:02:02.052065+0000","last_undegraded":"2024-04-25T15:02:02.052065+0000","last_fullsized":"2024-04-25T15:02:02.052065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196430+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:02:02.196430+0000","last_peered":"2024-04-25T15:02:02.196430+0000","last_clean":"2024-04-25T15:02:02.196430+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:02:02.196430+0000","last_undegraded":"2024-04-25T15:02:02.196430+0000","last_fullsized":"2024-04-25T15:02:02.196430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195909+0000","last_change":"2024-04-25T15:02:00.060948+0000","last_active":"2024-04-25T15:02:02.195909+0000","last_peered":"2024-04-25T15:02:02.195909+0000","last_clean":"2024-04-25T15:02:02.195909+0000","last_became_active":"2024-04-25T15:02:00.060583+0000","last_became_peered":"2024-04-25T15:02:00.060583+0000","last_unstale":"2024-04-25T15:02:02.195909+0000","last_undegraded":"2024-04-25T15:02:02.195909+0000","last_fullsized":"2024-04-25T15:02:02.195909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:10.061940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196695+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:02:02.196695+0000","last_peered":"2024-04-25T15:02:02.196695+0000","last_clean":"2024-04-25T15:02:02.196695+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:02:02.196695+0000","last_undegraded":"2024-04-25T15:02:02.196695+0000","last_fullsized":"2024-04-25T15:02:02.196695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.955895+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:02:02.955895+0000","last_peered":"2024-04-25T15:02:02.955895+0000","last_clean":"2024-04-25T15:02:02.955895+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:02:02.955895+0000","last_undegraded":"2024-04-25T15:02:02.955895+0000","last_fullsized":"2024-04-25T15:02:02.955895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196223+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:02:02.196223+0000","last_peered":"2024-04-25T15:02:02.196223+0000","last_clean":"2024-04-25T15:02:02.196223+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:02:02.196223+0000","last_undegraded":"2024-04-25T15:02:02.196223+0000","last_fullsized":"2024-04-25T15:02:02.196223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196716+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:02:02.196716+0000","last_peered":"2024-04-25T15:02:02.196716+0000","last_clean":"2024-04-25T15:02:02.196716+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:02:02.196716+0000","last_undegraded":"2024-04-25T15:02:02.196716+0000","last_fullsized":"2024-04-25T15:02:02.196716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196884+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:02:02.196884+0000","last_peered":"2024-04-25T15:02:02.196884+0000","last_clean":"2024-04-25T15:02:02.196884+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:02:02.196884+0000","last_undegraded":"2024-04-25T15:02:02.196884+0000","last_fullsized":"2024-04-25T15:02:02.196884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922288+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:02:01.922288+0000","last_peered":"2024-04-25T15:02:01.922288+0000","last_clean":"2024-04-25T15:02:01.922288+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:02:01.922288+0000","last_undegraded":"2024-04-25T15:02:01.922288+0000","last_fullsized":"2024-04-25T15:02:01.922288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195675+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:02:02.195675+0000","last_peered":"2024-04-25T15:02:02.195675+0000","last_clean":"2024-04-25T15:02:02.195675+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:02:02.195675+0000","last_undegraded":"2024-04-25T15:02:02.195675+0000","last_fullsized":"2024-04-25T15:02:02.195675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195607+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:02:02.195607+0000","last_peered":"2024-04-25T15:02:02.195607+0000","last_clean":"2024-04-25T15:02:02.195607+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:02:02.195607+0000","last_undegraded":"2024-04-25T15:02:02.195607+0000","last_fullsized":"2024-04-25T15:02:02.195607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196311+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:02:02.196311+0000","last_peered":"2024-04-25T15:02:02.196311+0000","last_clean":"2024-04-25T15:02:02.196311+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:02:02.196311+0000","last_undegraded":"2024-04-25T15:02:02.196311+0000","last_fullsized":"2024-04-25T15:02:02.196311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196945+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:02:02.196945+0000","last_peered":"2024-04-25T15:02:02.196945+0000","last_clean":"2024-04-25T15:02:02.196945+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:02:02.196945+0000","last_undegraded":"2024-04-25T15:02:02.196945+0000","last_fullsized":"2024-04-25T15:02:02.196945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922265+0000","last_change":"2024-04-25T15:02:00.944416+0000","last_active":"2024-04-25T15:02:01.922265+0000","last_peered":"2024-04-25T15:02:01.922265+0000","last_clean":"2024-04-25T15:02:01.922265+0000","last_became_active":"2024-04-25T15:02:00.944025+0000","last_became_peered":"2024-04-25T15:02:00.944025+0000","last_unstale":"2024-04-25T15:02:01.922265+0000","last_undegraded":"2024-04-25T15:02:01.922265+0000","last_fullsized":"2024-04-25T15:02:01.922265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:53.828823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196216+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:02:02.196216+0000","last_peered":"2024-04-25T15:02:02.196216+0000","last_clean":"2024-04-25T15:02:02.196216+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:02:02.196216+0000","last_undegraded":"2024-04-25T15:02:02.196216+0000","last_fullsized":"2024-04-25T15:02:02.196216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.054237+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:02:02.054237+0000","last_peered":"2024-04-25T15:02:02.054237+0000","last_clean":"2024-04-25T15:02:02.054237+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:02:02.054237+0000","last_undegraded":"2024-04-25T15:02:02.054237+0000","last_fullsized":"2024-04-25T15:02:02.054237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196120+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:02:02.196120+0000","last_peered":"2024-04-25T15:02:02.196120+0000","last_clean":"2024-04-25T15:02:02.196120+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:02:02.196120+0000","last_undegraded":"2024-04-25T15:02:02.196120+0000","last_fullsized":"2024-04-25T15:02:02.196120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.053244+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:02:02.053244+0000","last_peered":"2024-04-25T15:02:02.053244+0000","last_clean":"2024-04-25T15:02:02.053244+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:02:02.053244+0000","last_undegraded":"2024-04-25T15:02:02.053244+0000","last_fullsized":"2024-04-25T15:02:02.053244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.195675+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:02:02.195675+0000","last_peered":"2024-04-25T15:02:02.195675+0000","last_clean":"2024-04-25T15:02:02.195675+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:02:02.195675+0000","last_undegraded":"2024-04-25T15:02:02.195675+0000","last_fullsized":"2024-04-25T15:02:02.195675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:02.196937+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:02:02.196937+0000","last_peered":"2024-04-25T15:02:02.196937+0000","last_clean":"2024-04-25T15:02:02.196937+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:02:02.196937+0000","last_undegraded":"2024-04-25T15:02:02.196937+0000","last_fullsized":"2024-04-25T15:02:02.196937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:02:01.922061+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:02:01.922061+0000","last_peered":"2024-04-25T15:02:01.922061+0000","last_clean":"2024-04-25T15:02:01.922061+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:02:01.922061+0000","last_undegraded":"2024-04-25T15:02:01.922061+0000","last_fullsized":"2024-04-25T15:02:01.922061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":13,"num_read_kb":16,"num_write":52,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":33113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":452720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1064294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":453302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":453324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266727,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1544192,"data_stored":1043670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":452720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":454332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":452810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":454366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":452941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":452742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1552384,"data_stored":1043270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:02:09.807 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T15:02:09.808 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T15:02:09.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T15:02:10.313 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:10 smithi019 ceph-mon[28090]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:10.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:10 smithi019 ceph-mon[28090]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:10.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:10 smithi083 ceph-mon[32919]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:10.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:10 smithi083 ceph-mon[32919]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:10 smithi102 ceph-mon[33005]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:10 smithi102 ceph-mon[33005]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:11.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:11 smithi019 ceph-mon[28090]: pgmap v325: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-04-25T15:02:11.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:11 smithi083 ceph-mon[32919]: pgmap v325: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-04-25T15:02:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:11 smithi102 ceph-mon[33005]: pgmap v325: 97 pgs: 97 active+clean; 598 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-04-25T15:02:12.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 -- 172.21.15.19:0/3375294224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c105ee0 msgr2=0x7f494c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 --2- 172.21.15.19:0/3375294224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c105ee0 0x7f494c106340 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f49380099c0 tx=0x7f4938009cd0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 -- 172.21.15.19:0/3375294224 shutdown_connections 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 --2- 172.21.15.19:0/3375294224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c106880 0x7f494c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 --2- 172.21.15.19:0/3375294224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c104ce0 0x7f494c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 --2- 172.21.15.19:0/3375294224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c105ee0 0x7f494c106340 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 -- 172.21.15.19:0/3375294224 >> 172.21.15.19:0/3375294224 conn(0x7f494c1002a0 msgr2=0x7f494c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.898+0000 7f4953a00700 1 -- 172.21.15.19:0/3375294224 shutdown_connections 2024-04-25T15:02:12.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.899+0000 7f4953a00700 1 -- 172.21.15.19:0/3375294224 wait complete. 2024-04-25T15:02:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 Processor -- start 2024-04-25T15:02:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 -- start start 2024-04-25T15:02:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c106880 0x7f494c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f494c1107c0 con 0x7f494c106880 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f494c110640 con 0x7f494c104ce0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.900+0000 7f4953a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f494c110940 con 0x7f494c105ee0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f4951000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f4951000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:39868/0 (socket says 172.21.15.19:39868) 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f4951000700 1 -- 172.21.15.19:0/3449145150 learned_addr learned my addr 172.21.15.19:0/3449145150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:40626/0 (socket says 172.21.15.19:40626) 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f4951a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c106880 0x7f494c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 -- 172.21.15.19:0/3449145150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 msgr2=0x7f494c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 -- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c106880 msgr2=0x7f494c1a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c106880 0x7f494c1a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f494be00700 1 -- 172.21.15.19:0/3449145150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c1a29a0 con 0x7f494c105ee0 2024-04-25T15:02:12.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.901+0000 7f4951000700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f494be00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49380099c0 tx=0x7f4938003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380050c0 con 0x7f494c105ee0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493800f460 con 0x7f494c105ee0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c1a2c30 con 0x7f494c105ee0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938017670 con 0x7f494c105ee0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.902+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c1a30e0 con 0x7f494c105ee0 2024-04-25T15:02:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.904+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c050f80 con 0x7f494c105ee0 2024-04-25T15:02:12.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.906+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f493800f610 con 0x7f494c105ee0 2024-04-25T15:02:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.907+0000 7f4949600700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 0x7f49340a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.907+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f49380d1230 con 0x7f494c105ee0 2024-04-25T15:02:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.907+0000 7f4951000700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 0x7f49340a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.908+0000 7f4951000700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 0x7f49340a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4940005950 tx=0x7f4940009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:12.909+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4938013070 con 0x7f494c105ee0 2024-04-25T15:02:13.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.235+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f494c02e0f0 con 0x7f494c105ee0 2024-04-25T15:02:13.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.242+0000 7f4949600700 1 -- 172.21.15.19:0/3449145150 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f494c02e0f0 con 0x7f494c105ee0 2024-04-25T15:02:13.243 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T15:02:13.243 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQB1cCpmr5ssDhAA8czXP0gR4WVA6teJ+xXKSA== 2024-04-25T15:02:13.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 msgr2=0x7f49340a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:13.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 0x7f49340a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4940005950 tx=0x7f4940009260 comp rx=0 tx=0).stop 2024-04-25T15:02:13.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 msgr2=0x7f494c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:13.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49380099c0 tx=0x7f4938003af0 comp rx=0 tx=0).stop 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 shutdown_connections 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f49340a3e00 0x7f49340a62c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494c106880 0x7f494c1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f494c104ce0 0x7f494c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 --2- 172.21.15.19:0/3449145150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f494c105ee0 0x7f494c19b280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 >> 172.21.15.19:0/3449145150 conn(0x7f494c1002a0 msgr2=0x7f494c109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 shutdown_connections 2024-04-25T15:02:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:13.245+0000 7f4953a00700 1 -- 172.21.15.19:0/3449145150 wait complete. 2024-04-25T15:02:13.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:13 smithi019 ceph-mon[28090]: pgmap v326: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-25T15:02:13.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:13 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3449145150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:13 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:13 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:02:13.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:13 smithi083 ceph-mon[32919]: pgmap v326: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-25T15:02:13.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:13 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3449145150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:13 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:13 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:02:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:13 smithi102 ceph-mon[33005]: pgmap v326: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-25T15:02:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:13 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3449145150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:13 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:02:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:13 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:02:14.108 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T15:02:14.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T15:02:14.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T15:02:14.258 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T15:02:14.258 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T15:02:14.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T15:02:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:15.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:15 smithi019 ceph-mon[28090]: pgmap v327: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-25T15:02:15.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:15 smithi083 ceph-mon[32919]: pgmap v327: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-25T15:02:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:15 smithi102 ceph-mon[33005]: pgmap v327: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2024-04-25T15:02:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 -- 172.21.15.19:0/1761082646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a80108a30 msgr2=0x7f9a80108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 --2- 172.21.15.19:0/1761082646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a80108a30 0x7f9a80108e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a700098f0 tx=0x7f9a70009c00 comp rx=0 tx=0).stop 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 -- 172.21.15.19:0/1761082646 shutdown_connections 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 --2- 172.21.15.19:0/1761082646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 0x7f9a8010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 --2- 172.21.15.19:0/1761082646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a80108a30 0x7f9a80108e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 --2- 172.21.15.19:0/1761082646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a8006a750 0x7f9a8006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.092+0000 7f9a86400700 1 -- 172.21.15.19:0/1761082646 >> 172.21.15.19:0/1761082646 conn(0x7f9a800788f0 msgr2=0x7f9a80078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.093+0000 7f9a86400700 1 -- 172.21.15.19:0/1761082646 shutdown_connections 2024-04-25T15:02:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.093+0000 7f9a86400700 1 -- 172.21.15.19:0/1761082646 wait complete. 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.094+0000 7f9a86400700 1 Processor -- start 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.094+0000 7f9a86400700 1 -- start start 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 0x7f9a801996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80112c20 con 0x7f9a80108a30 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80112aa0 con 0x7f9a8006a750 2024-04-25T15:02:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a86400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80112da0 con 0x7f9a8006b0f0 2024-04-25T15:02:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a7fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a7f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a7f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:49960/0 (socket says 172.21.15.19:49960) 2024-04-25T15:02:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a7fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47610/0 (socket says 172.21.15.19:47610) 2024-04-25T15:02:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.095+0000 7f9a7f400700 1 -- 172.21.15.19:0/4028884791 learned_addr learned my addr 172.21.15.19:0/4028884791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7ea00700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 0x7f9a801996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 -- 172.21.15.19:0/4028884791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 msgr2=0x7f9a801996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 0x7f9a801996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 -- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 msgr2=0x7f9a8019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 -- 172.21.15.19:0/4028884791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8019e1c0 con 0x7f9a8006a750 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7fe00700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.096+0000 7f9a7f400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009230 tx=0x7f9a7000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.097+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70005140 con 0x7f9a8006a750 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.097+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a7000b560 con 0x7f9a8006a750 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.097+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70003970 con 0x7f9a8006a750 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.097+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8019e450 con 0x7f9a8006a750 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.097+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a801a5d30 con 0x7f9a8006a750 2024-04-25T15:02:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.098+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a80104170 con 0x7f9a8006a750 2024-04-25T15:02:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.100+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a7000ba50 con 0x7f9a8006a750 2024-04-25T15:02:17.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.101+0000 7f9a77e00700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 0x7f9a640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:17.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.102+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9a700d15f0 con 0x7f9a8006a750 2024-04-25T15:02:17.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.102+0000 7f9a7ea00700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 0x7f9a640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:17.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.102+0000 7f9a7ea00700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 0x7f9a640a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a80079030 tx=0x7f9a6c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:17.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.104+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a7005eff0 con 0x7f9a8006a750 2024-04-25T15:02:17.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:17 smithi019 ceph-mon[28090]: pgmap v328: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-25T15:02:17.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.401+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9a8002df80 con 0x7f9a8006a750 2024-04-25T15:02:17.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.407+0000 7f9a77e00700 1 -- 172.21.15.19:0/4028884791 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264993 (secure 0 0 0) 0x7f9a70013030 con 0x7f9a8006a750 2024-04-25T15:02:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.412+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 msgr2=0x7f9a640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.412+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 0x7f9a640a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a80079030 tx=0x7f9a6c009310 comp rx=0 tx=0).stop 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.412+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 msgr2=0x7f9a801991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.412+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009230 tx=0x7f9a7000fa70 comp rx=0 tx=0).stop 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 shutdown_connections 2024-04-25T15:02:17.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f9a640a3d50 0x7f9a640a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a80108a30 0x7f9a8019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a8006a750 0x7f9a801991b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 --2- 172.21.15.19:0/4028884791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a8006b0f0 0x7f9a801996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 >> 172.21.15.19:0/4028884791 conn(0x7f9a800788f0 msgr2=0x7f9a800ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 shutdown_connections 2024-04-25T15:02:17.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:17.413+0000 7f9a86400700 1 -- 172.21.15.19:0/4028884791 wait complete. 2024-04-25T15:02:17.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:17 smithi083 ceph-mon[32919]: pgmap v328: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-25T15:02:17.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:17 smithi102 ceph-mon[33005]: pgmap v328: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-25T15:02:18.385 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":983233775},{"type":"v1","addr":"172.21.15.19:6801","nonce":983233775}]},"active_addr":"172.21.15.19:6801/983233775","active_change":"2024-04-25T14:53:55.852781+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2796401471}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2746348246}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":71619263}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3395740379}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2691673531}]}]} 2024-04-25T15:02:18.390 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T15:02:18.390 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T15:02:18.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:02:18.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4028884791' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:02:18.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4028884791' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:02:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4028884791' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:02:18.902 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:19.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:19 smithi019 ceph-mon[28090]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-04-25T15:02:19.692 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:19 smithi083 ceph-mon[32919]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-04-25T15:02:19.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:19 smithi102 ceph-mon[33005]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-04-25T15:02:21.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 -- 172.21.15.19:0/2879029117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc109f70 msgr2=0x7f1fdc111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2879029117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc109f70 0x7f1fdc111b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009fc0 tx=0x7f1fd400b330 comp rx=0 tx=0).stop 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 -- 172.21.15.19:0/2879029117 shutdown_connections 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2879029117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc109f70 0x7f1fdc111b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2879029117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2879029117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc105840 0x7f1fdc109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 -- 172.21.15.19:0/2879029117 >> 172.21.15.19:0/2879029117 conn(0x7f1fdc100720 msgr2=0x7f1fdc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 -- 172.21.15.19:0/2879029117 shutdown_connections 2024-04-25T15:02:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.218+0000 7f1fe2800700 1 -- 172.21.15.19:0/2879029117 wait complete. 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.219+0000 7f1fe2800700 1 Processor -- start 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.219+0000 7f1fe2800700 1 -- start start 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc105840 0x7f1fdc19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 0x7f1fdc19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc114e00 con 0x7f1fdc104e90 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc114c80 con 0x7f1fdc109f70 2024-04-25T15:02:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fe2800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc114f80 con 0x7f1fdc105840 2024-04-25T15:02:21.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fdb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:21.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fdb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47616/0 (socket says 172.21.15.19:47616) 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fdb400700 1 -- 172.21.15.19:0/2631928431 learned_addr learned my addr 172.21.15.19:0/2631928431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fdbe00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 0x7f1fdc19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.220+0000 7f1fdaa00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc105840 0x7f1fdc19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 -- 172.21.15.19:0/2631928431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc105840 msgr2=0x7f1fdc19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc105840 0x7f1fdc19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 -- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 msgr2=0x7f1fdc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 -- 172.21.15.19:0/2631928431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc1a02b0 con 0x7f1fdc109f70 2024-04-25T15:02:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdb400700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fdbe00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 0x7f1fdc19fb70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4015000 tx=0x7f1fd400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4021070 con 0x7f1fdc109f70 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.221+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc1a0540 con 0x7f1fdc109f70 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.222+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd4007ac0 con 0x7f1fdc109f70 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.222+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4005040 con 0x7f1fdc109f70 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.222+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc1a7e20 con 0x7f1fdc109f70 2024-04-25T15:02:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.223+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc069730 con 0x7f1fdc109f70 2024-04-25T15:02:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.225+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1fd4024070 con 0x7f1fdc109f70 2024-04-25T15:02:21.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.226+0000 7f1fd3e00700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 0x7f1fc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:21.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.226+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1fd400f020 con 0x7f1fdc109f70 2024-04-25T15:02:21.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.226+0000 7f1fdb400700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 0x7f1fc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:21.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.227+0000 7f1fdb400700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 0x7f1fc00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc005950 tx=0x7f1fcc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:21.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.229+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fd402b020 con 0x7f1fdc109f70 2024-04-25T15:02:21.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.464+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fdc02df80 con 0x7f1fdc109f70 2024-04-25T15:02:21.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.466+0000 7f1fd3e00700 1 -- 172.21.15.19:0/2631928431 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22148 (secure 0 0 0) 0x7f1fd409e6d0 con 0x7f1fdc109f70 2024-04-25T15:02:21.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:21.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:01.913953+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:21.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 msgr2=0x7f1fc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:21.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 0x7f1fc00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc005950 tx=0x7f1fcc009310 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 msgr2=0x7f1fdc19fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 0x7f1fdc19fb70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4015000 tx=0x7f1fd400ccb0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 shutdown_connections 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fdc109f70 0x7f1fdc19fb70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1fc00a3cd0 0x7f1fc00a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fdc104e90 0x7f1fdc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 --2- 172.21.15.19:0/2631928431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fdc105840 0x7f1fdc19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 >> 172.21.15.19:0/2631928431 conn(0x7f1fdc100720 msgr2=0x7f1fdc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 shutdown_connections 2024-04-25T15:02:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:21.469+0000 7f1fe2800700 1 -- 172.21.15.19:0/2631928431 wait complete. 2024-04-25T15:02:21.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:21 smithi019 ceph-mon[28090]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s 2024-04-25T15:02:21.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:21 smithi083 ceph-mon[32919]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s 2024-04-25T15:02:21.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:21 smithi102 ceph-mon[33005]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s 2024-04-25T15:02:22.299 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T15:02:22.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:02:22.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:22 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2631928431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:22.782 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:22.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:22 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2631928431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:22 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2631928431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:23.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:23 smithi019 ceph-mon[28090]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-25T15:02:23.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:23 smithi083 ceph-mon[32919]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-25T15:02:23.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:23 smithi102 ceph-mon[33005]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-25T15:02:24.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 -- 172.21.15.19:0/4098453424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 msgr2=0x7f6e1c077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 --2- 172.21.15.19:0/4098453424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c077350 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009990 tx=0x7f6e0c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 -- 172.21.15.19:0/4098453424 shutdown_connections 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 --2- 172.21.15.19:0/4098453424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 --2- 172.21.15.19:0/4098453424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c077350 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 --2- 172.21.15.19:0/4098453424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c1133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 -- 172.21.15.19:0/4098453424 >> 172.21.15.19:0/4098453424 conn(0x7f6e1c100770 msgr2=0x7f6e1c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.993+0000 7f6e23400700 1 -- 172.21.15.19:0/4098453424 shutdown_connections 2024-04-25T15:02:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.994+0000 7f6e23400700 1 -- 172.21.15.19:0/4098453424 wait complete. 2024-04-25T15:02:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 Processor -- start 2024-04-25T15:02:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 -- start start 2024-04-25T15:02:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1192e0 con 0x7f6e1c076f70 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c119160 con 0x7f6e1c077890 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.995+0000 7f6e23400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c119460 con 0x7f6e1c113920 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e13e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54862/0 (socket says 172.21.15.19:54862) 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 -- 172.21.15.19:0/3502972294 learned_addr learned my addr 172.21.15.19:0/3502972294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e21400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 -- 172.21.15.19:0/3502972294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 msgr2=0x7f6e1c1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 -- 172.21.15.19:0/3502972294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 msgr2=0x7f6e1c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 -- 172.21.15.19:0/3502972294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c1a4740 con 0x7f6e1c076f70 2024-04-25T15:02:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e13e00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e21400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.996+0000 7f6e20a00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005910 tx=0x7f6e0c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.997+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c027070 con 0x7f6e1c076f70 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.997+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0c022470 con 0x7f6e1c076f70 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.997+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c1a49d0 con 0x7f6e1c076f70 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.997+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c017650 con 0x7f6e1c076f70 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.997+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c1ac2b0 con 0x7f6e1c076f70 2024-04-25T15:02:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:24.998+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1c069730 con 0x7f6e1c076f70 2024-04-25T15:02:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.001+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6e0c02a070 con 0x7f6e1c076f70 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.002+0000 7f6e1a000700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 0x7f6e000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.002+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6e0c0d1180 con 0x7f6e1c076f70 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.002+0000 7f6e13e00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 0x7f6e000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.003+0000 7f6e13e00700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 0x7f6e000a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c1041d0 tx=0x7f6e0400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.007+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e0c013070 con 0x7f6e1c076f70 2024-04-25T15:02:25.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.241+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e1c02df80 con 0x7f6e1c076f70 2024-04-25T15:02:25.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.245+0000 7f6e1a000700 1 -- 172.21.15.19:0/3502972294 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22148 (secure 0 0 0) 0x7f6e0c09db40 con 0x7f6e1c076f70 2024-04-25T15:02:25.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:02:25.249 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:01.913953+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:25.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.248+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 msgr2=0x7f6e000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.248+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 0x7f6e000a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c1041d0 tx=0x7f6e0400a380 comp rx=0 tx=0).stop 2024-04-25T15:02:25.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.248+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 msgr2=0x7f6e1c19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.248+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005910 tx=0x7f6e0c00b670 comp rx=0 tx=0).stop 2024-04-25T15:02:25.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 shutdown_connections 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e1c113920 0x7f6e1c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6e000a3d50 0x7f6e000a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c076f70 0x7f6e1c19f730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 --2- 172.21.15.19:0/3502972294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1c077890 0x7f6e1c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 >> 172.21.15.19:0/3502972294 conn(0x7f6e1c100770 msgr2=0x7f6e1c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 shutdown_connections 2024-04-25T15:02:25.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:25.249+0000 7f6e23400700 1 -- 172.21.15.19:0/3502972294 wait complete. 2024-04-25T15:02:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:25 smithi019 ceph-mon[28090]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-25T15:02:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:25 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3502972294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:25.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:25 smithi083 ceph-mon[32919]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-25T15:02:25.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:25 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3502972294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:25 smithi102 ceph-mon[33005]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-25T15:02:25.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:25 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3502972294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T15:02:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T15:02:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T15:02:26.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T15:02:26.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T15:02:26.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T15:02:26.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T15:02:26.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T15:02:26.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T15:02:26.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T15:02:26.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T15:02:26.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T15:02:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.342 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.351 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.355 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.356 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:27.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:27 smithi019 ceph-mon[28090]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-25T15:02:27.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:27 smithi083 ceph-mon[32919]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-25T15:02:27.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:27 smithi102 ceph-mon[33005]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-25T15:02:29.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:29 smithi019 ceph-mon[28090]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-04-25T15:02:29.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:29 smithi083 ceph-mon[32919]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-04-25T15:02:29.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:29 smithi102 ceph-mon[33005]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-04-25T15:02:31.524 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:31 smithi019 ceph-mon[28090]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:31.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:31 smithi083 ceph-mon[32919]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:31.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:31 smithi102 ceph-mon[33005]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:33.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:33 smithi019 ceph-mon[28090]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:33.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:33 smithi083 ceph-mon[32919]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:33 smithi102 ceph-mon[33005]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:35.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:35 smithi019 ceph-mon[28090]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:35.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:35 smithi083 ceph-mon[32919]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:35.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:35 smithi102 ceph-mon[33005]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-25T15:02:37.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:37 smithi019 ceph-mon[28090]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:02:37.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:37 smithi083 ceph-mon[32919]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:02:37.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:37 smithi102 ceph-mon[33005]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:02:39.461 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:39 smithi019 ceph-mon[28090]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:02:39.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:39 smithi083 ceph-mon[32919]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:02:39.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:39 smithi102 ceph-mon[33005]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:02:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.815+0000 7fa43d400700 1 -- 172.21.15.19:0/3269787391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438102ad0 msgr2=0x7fa438102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.815+0000 7fa43d400700 1 --2- 172.21.15.19:0/3269787391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438102ad0 0x7fa438102f50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa424009990 tx=0x7fa424009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 -- 172.21.15.19:0/3269787391 shutdown_connections 2024-04-25T15:02:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 --2- 172.21.15.19:0/3269787391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438103490 0x7fa43810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 --2- 172.21.15.19:0/3269787391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa438108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 --2- 172.21.15.19:0/3269787391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438102ad0 0x7fa438102f50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 -- 172.21.15.19:0/3269787391 >> 172.21.15.19:0/3269787391 conn(0x7fa438079c30 msgr2=0x7fa43807a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 -- 172.21.15.19:0/3269787391 shutdown_connections 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.816+0000 7fa43d400700 1 -- 172.21.15.19:0/3269787391 wait complete. 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.817+0000 7fa43d400700 1 Processor -- start 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.817+0000 7fa43d400700 1 -- start start 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102ad0 0x7fa438074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438103490 0x7fa438075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa4380757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa438112cd0 con 0x7fa438108aa0 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa438112b50 con 0x7fa438103490 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa43d400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa438112e50 con 0x7fa438102ad0 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa437200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa4380757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa437200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa4380757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52760/0 (socket says 172.21.15.19:52760) 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa437200700 1 -- 172.21.15.19:0/1614332738 learned_addr learned my addr 172.21.15.19:0/1614332738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa436800700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102ad0 0x7fa438074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.818+0000 7fa435e00700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438103490 0x7fa438075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 -- 172.21.15.19:0/1614332738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438103490 msgr2=0x7fa438075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa438103490 0x7fa438075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 -- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 msgr2=0x7fa4380757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa4380757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4381a7750 con 0x7fa438102ad0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa437200700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa438108aa0 0x7fa4380757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa436800700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102ad0 0x7fa438074d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa42800e9f0 tx=0x7fa42800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800c9c0 con 0x7fa438102ad0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42800cb60 con 0x7fa438102ad0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4381a7a40 con 0x7fa438102ad0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.819+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428018680 con 0x7fa438102ad0 2024-04-25T15:02:39.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.820+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4381a7f80 con 0x7fa438102ad0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.820+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4380630d0 con 0x7fa438102ad0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.822+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa428010430 con 0x7fa438102ad0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa42f400700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3cd0 0x7fa41c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa4280d0ba0 con 0x7fa438102ad0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa435e00700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3cd0 0x7fa41c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa42f400700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] conn(0x7fa41c0ad4b0 0x7fa41c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa42800be50 con 0x7fa41c0ad4b0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.824+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa4280d0f50 con 0x7fa438102ad0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.825+0000 7fa437200700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] conn(0x7fa41c0ad4b0 0x7fa41c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.825+0000 7fa435e00700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3cd0 0x7fa41c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa43807a310 tx=0x7fa42400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.825+0000 7fa437200700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] conn(0x7fa41c0ad4b0 0x7fa41c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.826+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== osd.11 v2:172.21.15.102:6824/2593361750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa42800be50 con 0x7fa41c0ad4b0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.844+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 --> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4380634d0 con 0x7fa41c0ad4b0 2024-04-25T15:02:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.845+0000 7fa42f400700 1 -- 172.21.15.19:0/1614332738 <== osd.11 v2:172.21.15.102:6824/2593361750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa4380634d0 con 0x7fa41c0ad4b0 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] conn(0x7fa41c0ad4b0 msgr2=0x7fa41c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] conn(0x7fa41c0ad4b0 0x7fa41c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3cd0 msgr2=0x7fa41c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3cd0 0x7fa41c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa43807a310 tx=0x7fa42400b260 comp rx=0 tx=0).stop 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102ad0 msgr2=0x7fa438074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 --2- 172.21.15.19:0/1614332738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa438102ad0 0x7fa438074d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa42800e9f0 tx=0x7fa42800ed00 comp rx=0 tx=0).stop 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa437200700 1 -- 172.21.15.19:0/1614332738 reap_dead start 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 shutdown_connections 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 >> 172.21.15.19:0/1614332738 conn(0x7fa438079c30 msgr2=0x7fa438100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:39.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 shutdown_connections 2024-04-25T15:02:39.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:39.846+0000 7fa43d400700 1 -- 172.21.15.19:0/1614332738 wait complete. 2024-04-25T15:02:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.474+0000 7f47c8e00700 1 -- 172.21.15.19:0/2936563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4104ec0 msgr2=0x7f47c41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.474+0000 7f47c8e00700 1 --2- 172.21.15.19:0/2936563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4104ec0 0x7f47c41052a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009990 tx=0x7f47b4009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 -- 172.21.15.19:0/2936563967 shutdown_connections 2024-04-25T15:02:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 --2- 172.21.15.19:0/2936563967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4109fa0 0x7f47c4111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 --2- 172.21.15.19:0/2936563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4104ec0 0x7f47c41052a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 --2- 172.21.15.19:0/2936563967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4105870 0x7f47c41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 -- 172.21.15.19:0/2936563967 >> 172.21.15.19:0/2936563967 conn(0x7f47c4100750 msgr2=0x7f47c4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 -- 172.21.15.19:0/2936563967 shutdown_connections 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.475+0000 7f47c8e00700 1 -- 172.21.15.19:0/2936563967 wait complete. 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.476+0000 7f47c8e00700 1 Processor -- start 2024-04-25T15:02:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.476+0000 7f47c8e00700 1 -- start start 2024-04-25T15:02:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 0x7f47c419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4105870 0x7f47c419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 0x7f47c419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4114ea0 con 0x7f47c4109fa0 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4114d20 con 0x7f47c4105870 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c8e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4115020 con 0x7f47c4104ec0 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 0x7f47c419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c1e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 0x7f47c419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c1400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4105870 0x7f47c419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 0x7f47c419fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52796/0 (socket says 172.21.15.19:52796) 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c1e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 0x7f47c419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:37712/0 (socket says 172.21.15.19:37712) 2024-04-25T15:02:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.477+0000 7f47c2800700 1 -- 172.21.15.19:0/781890330 learned_addr learned my addr 172.21.15.19:0/781890330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 -- 172.21.15.19:0/781890330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4105870 msgr2=0x7f47c419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4105870 0x7f47c419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 -- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 msgr2=0x7f47c419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 0x7f47c419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c41a0360 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1400700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c4105870 0x7f47c419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c2800700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c4109fa0 0x7f47c419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c1e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 0x7f47c419b350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098e0 tx=0x7f47b40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4027070 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.478+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c41a05f0 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.479+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a7ed0 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.479+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b400b610 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.479+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40173f0 con 0x7f47c4104ec0 2024-04-25T15:02:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.479+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47a4001030 con 0x7f47c4104ec0 2024-04-25T15:02:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.481+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47b402a070 con 0x7f47c4104ec0 2024-04-25T15:02:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47baa00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f47a80a3cd0 0x7f47a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f47b40d10e0 con 0x7f47c4104ec0 2024-04-25T15:02:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47c1400700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f47a80a3cd0 0x7f47a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47baa00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] conn(0x7f47a80ad380 0x7f47a80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47b401de50 con 0x7f47a80ad380 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.483+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f47b40d8050 con 0x7f47c4104ec0 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.484+0000 7f47c2800700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] conn(0x7f47a80ad380 0x7f47a80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.484+0000 7f47c1400700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f47a80a3cd0 0x7f47a80a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009680 tx=0x7f47b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.484+0000 7f47c2800700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] conn(0x7f47a80ad380 0x7f47a80af7a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.485+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== osd.3 v2:172.21.15.19:6810/3094191876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47b401de50 con 0x7f47a80ad380 2024-04-25T15:02:40.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.503+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 --> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47a4002e60 con 0x7f47a80ad380 2024-04-25T15:02:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.504+0000 7f47baa00700 1 -- 172.21.15.19:0/781890330 <== osd.3 v2:172.21.15.19:6810/3094191876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f47a4002e60 con 0x7f47a80ad380 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.504+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] conn(0x7f47a80ad380 msgr2=0x7f47a80af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.504+0000 7f47c8e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] conn(0x7f47a80ad380 0x7f47a80af7a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f47a80a3cd0 msgr2=0x7f47a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f47a80a3cd0 0x7f47a80a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009680 tx=0x7f47b0009260 comp rx=0 tx=0).stop 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 msgr2=0x7f47c419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 --2- 172.21.15.19:0/781890330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c4104ec0 0x7f47c419b350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098e0 tx=0x7f47b40057c0 comp rx=0 tx=0).stop 2024-04-25T15:02:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c2800700 1 -- 172.21.15.19:0/781890330 reap_dead start 2024-04-25T15:02:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 shutdown_connections 2024-04-25T15:02:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 >> 172.21.15.19:0/781890330 conn(0x7f47c4100750 msgr2=0x7f47c4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 shutdown_connections 2024-04-25T15:02:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.505+0000 7f47c8e00700 1 -- 172.21.15.19:0/781890330 wait complete. 2024-04-25T15:02:41.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.294+0000 7fd8d5200700 1 -- 172.21.15.19:0/1084914028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0104450 msgr2=0x7fd8d010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.294+0000 7fd8d5200700 1 --2- 172.21.15.19:0/1084914028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0104450 0x7fd8d010c940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009990 tx=0x7fd8b8009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.294+0000 7fd8d5200700 1 -- 172.21.15.19:0/1084914028 shutdown_connections 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.294+0000 7fd8d5200700 1 --2- 172.21.15.19:0/1084914028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d010cf60 0x7fd8d010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.295+0000 7fd8d5200700 1 --2- 172.21.15.19:0/1084914028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.295+0000 7fd8d5200700 1 --2- 172.21.15.19:0/1084914028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0104450 0x7fd8d010c940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.295+0000 7fd8d5200700 1 -- 172.21.15.19:0/1084914028 >> 172.21.15.19:0/1084914028 conn(0x7fd8d00fd360 msgr2=0x7fd8d00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.295+0000 7fd8d5200700 1 -- 172.21.15.19:0/1084914028 shutdown_connections 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.295+0000 7fd8d5200700 1 -- 172.21.15.19:0/1084914028 wait complete. 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.296+0000 7fd8d5200700 1 Processor -- start 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.296+0000 7fd8d5200700 1 -- start start 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.296+0000 7fd8d5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8d5200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0104450 0x7fd8d0101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:41.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8d5200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8d010cf60 0x7fd8d0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8d5200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0112b90 con 0x7fd8d0103b30 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8d5200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0112a10 con 0x7fd8d010cf60 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8d5200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0112d10 con 0x7fd8d0104450 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52812/0 (socket says 172.21.15.19:52812) 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 -- 172.21.15.19:0/889134114 learned_addr learned my addr 172.21.15.19:0/889134114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8c7e00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0104450 0x7fd8d0101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cfe00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8d010cf60 0x7fd8d0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 -- 172.21.15.19:0/889134114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0104450 msgr2=0x7fd8d0101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0104450 0x7fd8d0101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 -- 172.21.15.19:0/889134114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8d010cf60 msgr2=0x7fd8d0101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8d010cf60 0x7fd8d0101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8cf400700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d01aba90 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.297+0000 7fd8c7e00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0104450 0x7fd8d0101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8cf400700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000e9a0 tx=0x7fd8c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000ca10 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8c000cbb0 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d01abd20 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0018680 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.298+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01ac1d0 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.299+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8d00630d0 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.302+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8c0010430 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.303+0000 7fd8ccc00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd8b40a3d50 0x7fd8b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.303+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd8c00d0b50 con 0x7fd8d0103b30 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.303+0000 7fd8c7e00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd8b40a3d50 0x7fd8b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.303+0000 7fd8ccc00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] conn(0x7fd8b40ad530 0x7fd8b40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.304+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8c0019ec0 con 0x7fd8b40ad530 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.304+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd8c00d0f00 con 0x7fd8d0103b30 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.304+0000 7fd8cfe00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] conn(0x7fd8b40ad530 0x7fd8b40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.304+0000 7fd8c7e00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd8b40a3d50 0x7fd8b40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8d019d220 tx=0x7fd8b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.304+0000 7fd8cfe00700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] conn(0x7fd8b40ad530 0x7fd8b40af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.306+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== osd.0 v2:172.21.15.19:6802/1027909993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8c0019ec0 con 0x7fd8b40ad530 2024-04-25T15:02:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.324+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 --> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8d0063550 con 0x7fd8b40ad530 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.324+0000 7fd8ccc00700 1 -- 172.21.15.19:0/889134114 <== osd.0 v2:172.21.15.19:6802/1027909993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd8d0063550 con 0x7fd8b40ad530 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] conn(0x7fd8b40ad530 msgr2=0x7fd8b40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] conn(0x7fd8b40ad530 0x7fd8b40af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd8b40a3d50 msgr2=0x7fd8b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd8b40a3d50 0x7fd8b40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8d019d220 tx=0x7fd8b801a040 comp rx=0 tx=0).stop 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 msgr2=0x7fd8d0103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 --2- 172.21.15.19:0/889134114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8d0103b30 0x7fd8d0103380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000e9a0 tx=0x7fd8c000ecb0 comp rx=0 tx=0).stop 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8cfe00700 1 -- 172.21.15.19:0/889134114 reap_dead start 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 shutdown_connections 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 >> 172.21.15.19:0/889134114 conn(0x7fd8d00fd360 msgr2=0x7fd8d00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 shutdown_connections 2024-04-25T15:02:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:41.325+0000 7fd8d5200700 1 -- 172.21.15.19:0/889134114 wait complete. 2024-04-25T15:02:41.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:41 smithi019 ceph-mon[28090]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:41 smithi083 ceph-mon[32919]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:41.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:41 smithi102 ceph-mon[33005]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 -- 172.21.15.19:0/2383049766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424109fc0 msgr2=0x7f2424111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 --2- 172.21.15.19:0/2383049766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424109fc0 0x7f2424111b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b0d0 tx=0x7f241c00b3e0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 -- 172.21.15.19:0/2383049766 shutdown_connections 2024-04-25T15:02:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 --2- 172.21.15.19:0/2383049766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424109fc0 0x7f2424111b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 --2- 172.21.15.19:0/2383049766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424104ee0 0x7f24241052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 --2- 172.21.15.19:0/2383049766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2424105890 0x7f24241098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.690+0000 7f242b200700 1 -- 172.21.15.19:0/2383049766 >> 172.21.15.19:0/2383049766 conn(0x7f2424100770 msgr2=0x7f2424102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.691+0000 7f242b200700 1 -- 172.21.15.19:0/2383049766 shutdown_connections 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.691+0000 7f242b200700 1 -- 172.21.15.19:0/2383049766 wait complete. 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 Processor -- start 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 -- start start 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 0x7f242419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424105890 0x7f242419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2424109fc0 0x7f242419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2424114ec0 con 0x7f2424104ee0 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2424114d40 con 0x7f2424105890 2024-04-25T15:02:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.692+0000 7f242b200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2424115040 con 0x7f2424109fc0 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 0x7f242419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 0x7f242419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52824/0 (socket says 172.21.15.19:52824) 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 -- 172.21.15.19:0/4104160603 learned_addr learned my addr 172.21.15.19:0/4104160603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423400700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424105890 0x7f242419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2429200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2424109fc0 0x7f242419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 -- 172.21.15.19:0/4104160603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2424109fc0 msgr2=0x7f242419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2424109fc0 0x7f242419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 -- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424105890 msgr2=0x7f242419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424105890 0x7f242419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24241a0380 con 0x7f2424104ee0 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423400700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2424105890 0x7f242419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.693+0000 7f2423e00700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 0x7f242419b370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f241400e9a0 tx=0x7f241400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.694+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400ca10 con 0x7f2424104ee0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.694+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f241400cbb0 con 0x7f2424104ee0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.694+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414018680 con 0x7f2424104ee0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.694+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241a0670 con 0x7f2424104ee0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.694+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241a7ef0 con 0x7f2424104ee0 2024-04-25T15:02:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.695+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24240630d0 con 0x7f2424104ee0 2024-04-25T15:02:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.697+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2414010430 con 0x7f2424104ee0 2024-04-25T15:02:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2420c00700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f24080a3d50 0x7f24080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f24140d0b70 con 0x7f2424104ee0 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2423400700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f24080a3d50 0x7f24080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2420c00700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] conn(0x7f24080ad530 0x7f24080af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2414016ec0 con 0x7f24080ad530 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.699+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f24140d0f20 con 0x7f2424104ee0 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.700+0000 7f2429200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] conn(0x7f24080ad530 0x7f24080af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.700+0000 7f2423400700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f24080a3d50 0x7f24080a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f242410b5a0 tx=0x7f2410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.700+0000 7f2429200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] conn(0x7f24080ad530 0x7f24080af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.701+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== osd.1 v2:172.21.15.83:6800/3419816052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2414016ec0 con 0x7f24080ad530 2024-04-25T15:02:42.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.719+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 --> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24240635a0 con 0x7f24080ad530 2024-04-25T15:02:42.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.720+0000 7f2420c00700 1 -- 172.21.15.19:0/4104160603 <== osd.1 v2:172.21.15.83:6800/3419816052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f24240635a0 con 0x7f24080ad530 2024-04-25T15:02:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] conn(0x7f24080ad530 msgr2=0x7f24080af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] conn(0x7f24080ad530 0x7f24080af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f24080a3d50 msgr2=0x7f24080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f24080a3d50 0x7f24080a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f242410b5a0 tx=0x7f2410009260 comp rx=0 tx=0).stop 2024-04-25T15:02:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 msgr2=0x7f242419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 --2- 172.21.15.19:0/4104160603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2424104ee0 0x7f242419b370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f241400e9a0 tx=0x7f241400ecb0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f2429200700 1 -- 172.21.15.19:0/4104160603 reap_dead start 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 shutdown_connections 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.721+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 >> 172.21.15.19:0/4104160603 conn(0x7f2424100770 msgr2=0x7f24241012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.722+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 shutdown_connections 2024-04-25T15:02:42.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:42.722+0000 7f242b200700 1 -- 172.21.15.19:0/4104160603 wait complete. 2024-04-25T15:02:42.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 -- 172.21.15.19:0/1237586504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60104ee0 msgr2=0x7fbf601052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 --2- 172.21.15.19:0/1237586504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60104ee0 0x7fbf601052c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009990 tx=0x7fbf50009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 -- 172.21.15.19:0/1237586504 shutdown_connections 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 --2- 172.21.15.19:0/1237586504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60109fc0 0x7fbf60111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 --2- 172.21.15.19:0/1237586504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60104ee0 0x7fbf601052c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 --2- 172.21.15.19:0/1237586504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 0x7fbf601098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.320+0000 7fbf66000700 1 -- 172.21.15.19:0/1237586504 >> 172.21.15.19:0/1237586504 conn(0x7fbf60100770 msgr2=0x7fbf60102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.321+0000 7fbf66000700 1 -- 172.21.15.19:0/1237586504 shutdown_connections 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.321+0000 7fbf66000700 1 -- 172.21.15.19:0/1237586504 wait complete. 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.322+0000 7fbf66000700 1 Processor -- start 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.322+0000 7fbf66000700 1 -- start start 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 0x7fbf6019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 0x7fbf6019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60109fc0 0x7fbf6019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60114e50 con 0x7fbf60104ee0 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60114cd0 con 0x7fbf60109fc0 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf66000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60114fd0 con 0x7fbf60105890 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf5f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 0x7fbf6019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf5f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 0x7fbf6019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52784/0 (socket says 172.21.15.19:52784) 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf5f400700 1 -- 172.21.15.19:0/1166280237 learned_addr learned my addr 172.21.15.19:0/1166280237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf5ea00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 0x7fbf6019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.323+0000 7fbf5fe00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60109fc0 0x7fbf6019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 -- 172.21.15.19:0/1166280237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 msgr2=0x7fbf6019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 0x7fbf6019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 -- 172.21.15.19:0/1166280237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60109fc0 msgr2=0x7fbf6019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60109fc0 0x7fbf6019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf601a0370 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5fe00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf60109fc0 0x7fbf6019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5ea00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf60105890 0x7fbf6019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.324+0000 7fbf5f400700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 0x7fbf6019b360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbf500057c0 tx=0x7fbf50005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.325+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50026070 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.325+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50021410 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.325+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf601a0600 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.325+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601a7ee0 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.325+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000fb40 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.326+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf60063150 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.328+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf50009d10 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.330+0000 7fbf57e00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbf440a3cd0 0x7fbf440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.330+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbf500d1090 con 0x7fbf60104ee0 2024-04-25T15:02:42.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.330+0000 7fbf5ea00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbf440a3cd0 0x7fbf440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.330+0000 7fbf57e00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] conn(0x7fbf440ad4b0 0x7fbf440af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.331+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf5001ae50 con 0x7fbf440ad4b0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.331+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbf500d1440 con 0x7fbf60104ee0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.331+0000 7fbf5fe00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] conn(0x7fbf440ad4b0 0x7fbf440af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.331+0000 7fbf5ea00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbf440a3cd0 0x7fbf440a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009680 tx=0x7fbf4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.332+0000 7fbf5fe00700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] conn(0x7fbf440ad4b0 0x7fbf440af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.333+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== osd.9 v2:172.21.15.19:6826/2037522312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf5001ae50 con 0x7fbf440ad4b0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.352+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 --> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf600635d0 con 0x7fbf440ad4b0 2024-04-25T15:02:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf57e00700 1 -- 172.21.15.19:0/1166280237 <== osd.9 v2:172.21.15.19:6826/2037522312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf600635d0 con 0x7fbf440ad4b0 2024-04-25T15:02:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] conn(0x7fbf440ad4b0 msgr2=0x7fbf440af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] conn(0x7fbf440ad4b0 0x7fbf440af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbf440a3cd0 msgr2=0x7fbf440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbf440a3cd0 0x7fbf440a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009680 tx=0x7fbf4c009260 comp rx=0 tx=0).stop 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 msgr2=0x7fbf6019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf66000700 1 --2- 172.21.15.19:0/1166280237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf60104ee0 0x7fbf6019b360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbf500057c0 tx=0x7fbf50005ef0 comp rx=0 tx=0).stop 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.353+0000 7fbf5fe00700 1 -- 172.21.15.19:0/1166280237 reap_dead start 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.354+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 shutdown_connections 2024-04-25T15:02:42.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.354+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 >> 172.21.15.19:0/1166280237 conn(0x7fbf60100770 msgr2=0x7fbf60109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.354+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 shutdown_connections 2024-04-25T15:02:42.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:40.354+0000 7fbf66000700 1 -- 172.21.15.19:0/1166280237 wait complete. 2024-04-25T15:02:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:43 smithi083 ceph-mon[32919]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:43.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:43 smithi019 ceph-mon[28090]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:43 smithi102 ceph-mon[33005]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:44.149 INFO:teuthology.orchestra.run.smithi019.stdout:300647710753 2024-04-25T15:02:44.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T15:02:45.603 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:45 smithi019 ceph-mon[28090]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:02:45.776 INFO:teuthology.orchestra.run.smithi019.stdout:356482285589 2024-04-25T15:02:45.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T15:02:45.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:45 smithi083 ceph-mon[32919]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:02:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:45 smithi102 ceph-mon[33005]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 -- 172.21.15.19:0/928208340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960109fa0 msgr2=0x7ff960111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 --2- 172.21.15.19:0/928208340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960109fa0 0x7ff960111b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b0d0 tx=0x7ff95800b3e0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 -- 172.21.15.19:0/928208340 shutdown_connections 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 --2- 172.21.15.19:0/928208340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960109fa0 0x7ff960111b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 --2- 172.21.15.19:0/928208340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960104ec0 0x7ff9601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 --2- 172.21.15.19:0/928208340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 0x7ff9601098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 -- 172.21.15.19:0/928208340 >> 172.21.15.19:0/928208340 conn(0x7ff960100750 msgr2=0x7ff960102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 -- 172.21.15.19:0/928208340 shutdown_connections 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.330+0000 7ff968600700 1 -- 172.21.15.19:0/928208340 wait complete. 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 Processor -- start 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 -- start start 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960104ec0 0x7ff96019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 0x7ff96019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960109fa0 0x7ff96019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff960114e90 con 0x7ff960104ec0 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff960114d10 con 0x7ff960105870 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff968600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff960115010 con 0x7ff960109fa0 2024-04-25T15:02:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.332+0000 7ff965200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 0x7ff96019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff965200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 0x7ff96019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:52944/0 (socket says 172.21.15.19:52944) 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff965200700 1 -- 172.21.15.19:0/2740586460 learned_addr learned my addr 172.21.15.19:0/2740586460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960109fa0 0x7ff96019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff965c00700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960104ec0 0x7ff96019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 -- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 msgr2=0x7ff96019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff960105870 0x7ff96019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 -- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960104ec0 msgr2=0x7ff96019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960104ec0 0x7ff96019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9601a02b0 con 0x7ff960109fa0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff965c00700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960104ec0 0x7ff96019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.333+0000 7ff966600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960109fa0 0x7ff96019fb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b770 tx=0x7ff958007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.334+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95801f070 con 0x7ff960109fa0 2024-04-25T15:02:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.334+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff958004c50 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.334+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9601a0540 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.334+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95801b690 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.334+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9601a7e20 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.335+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff960101bd0 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.337+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff958022070 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.338+0000 7ff95e000700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3cd0 0x7ff9480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff95800f020 con 0x7ff960109fa0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff965c00700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3cd0 0x7ff9480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff95e000700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] conn(0x7ff9480ad4b0 0x7ff9480af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9480aff80 con 0x7ff9480ad4b0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff9580d1b10 con 0x7ff960109fa0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.339+0000 7ff965200700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] conn(0x7ff9480ad4b0 0x7ff9480af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.340+0000 7ff965c00700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3cd0 0x7ff9480a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff954009cb0 tx=0x7ff954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.340+0000 7ff965200700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] conn(0x7ff9480ad4b0 0x7ff9480af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.341+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== osd.10 v2:172.21.15.83:6824/2320025871 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9480aff80 con 0x7ff9480ad4b0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.360+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 --> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9601063a0 con 0x7ff9480ad4b0 2024-04-25T15:02:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff95e000700 1 -- 172.21.15.19:0/2740586460 <== osd.10 v2:172.21.15.83:6824/2320025871 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9601063a0 con 0x7ff9480ad4b0 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] conn(0x7ff9480ad4b0 msgr2=0x7ff9480af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] conn(0x7ff9480ad4b0 0x7ff9480af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3cd0 msgr2=0x7ff9480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3cd0 0x7ff9480a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff954009cb0 tx=0x7ff954009260 comp rx=0 tx=0).stop 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960109fa0 msgr2=0x7ff96019fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.361+0000 7ff968600700 1 --2- 172.21.15.19:0/2740586460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff960109fa0 0x7ff96019fb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b770 tx=0x7ff958007980 comp rx=0 tx=0).stop 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.362+0000 7ff966600700 1 -- 172.21.15.19:0/2740586460 reap_dead start 2024-04-25T15:02:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.362+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 shutdown_connections 2024-04-25T15:02:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.362+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 >> 172.21.15.19:0/2740586460 conn(0x7ff960100750 msgr2=0x7ff960101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.362+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 shutdown_connections 2024-04-25T15:02:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:43.362+0000 7ff968600700 1 -- 172.21.15.19:0/2740586460 wait complete. 2024-04-25T15:02:47.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:47 smithi019 ceph-mon[28090]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:02:47.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:47 smithi083 ceph-mon[32919]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:02:47.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:47 smithi102 ceph-mon[33005]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:02:48.113 INFO:teuthology.orchestra.run.smithi019.stdout:34359738453 2024-04-25T15:02:48.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 -- 172.21.15.19:0/3360135385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068109fa0 msgr2=0x7f9068111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 --2- 172.21.15.19:0/3360135385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068109fa0 0x7f9068111b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f906000b0d0 tx=0x7f906000b3e0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 -- 172.21.15.19:0/3360135385 shutdown_connections 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 --2- 172.21.15.19:0/3360135385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068109fa0 0x7f9068111b70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 --2- 172.21.15.19:0/3360135385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9068104ec0 0x7f90681052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 --2- 172.21.15.19:0/3360135385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068105870 0x7f90681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.404+0000 7f9070200700 1 -- 172.21.15.19:0/3360135385 >> 172.21.15.19:0/3360135385 conn(0x7f9068100750 msgr2=0x7f9068102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.405+0000 7f9070200700 1 -- 172.21.15.19:0/3360135385 shutdown_connections 2024-04-25T15:02:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.405+0000 7f9070200700 1 -- 172.21.15.19:0/3360135385 wait complete. 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.406+0000 7f9070200700 1 Processor -- start 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.406+0000 7f9070200700 1 -- start start 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.406+0000 7f9070200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 0x7f906819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f9070200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9068105870 0x7f906819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f9070200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068109fa0 0x7f906819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f9070200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9068114eb0 con 0x7f9068104ec0 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f9070200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9068114d30 con 0x7f9068109fa0 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f9070200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9068115030 con 0x7f9068105870 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 0x7f906819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 0x7f906819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38638/0 (socket says 172.21.15.19:38638) 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 -- 172.21.15.19:0/1863537325 learned_addr learned my addr 172.21.15.19:0/1863537325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906e200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068109fa0 0x7f906819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906ce00700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9068105870 0x7f906819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:48.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 -- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9068105870 msgr2=0x7f906819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9068105870 0x7f906819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 -- 172.21.15.19:0/1863537325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068109fa0 msgr2=0x7f906819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068109fa0 0x7f906819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906d800700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a02d0 con 0x7f9068104ec0 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.407+0000 7f906e200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9068109fa0 0x7f906819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f906d800700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 0x7f906819b2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e9c0 tx=0x7f905c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c00cbd0 con 0x7f9068104ec0 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c00ee60 con 0x7f9068104ec0 2024-04-25T15:02:48.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a05c0 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0055c0 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.408+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a7e40 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.409+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9068101b50 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.411+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f905c004750 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f9066000700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90500a3d50 0x7f90500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f905c0d0ac0 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f906ce00700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90500a3d50 0x7f90500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f9066000700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] conn(0x7f90500ad530 0x7f90500af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90500b0000 con 0x7f90500ad530 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f905c0d0e70 con 0x7f9068104ec0 2024-04-25T15:02:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.413+0000 7f906e200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] conn(0x7f90500ad530 0x7f90500af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.414+0000 7f906ce00700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90500a3d50 0x7f90500a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f906810b4a0 tx=0x7f9054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.414+0000 7f906e200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] conn(0x7f90500ad530 0x7f90500af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.415+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== osd.5 v2:172.21.15.102:6808/4087768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90500b0000 con 0x7f90500ad530 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.433+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 --> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90681063a0 con 0x7f90500ad530 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.434+0000 7f9066000700 1 -- 172.21.15.19:0/1863537325 <== osd.5 v2:172.21.15.102:6808/4087768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f90681063a0 con 0x7f90500ad530 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.434+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] conn(0x7f90500ad530 msgr2=0x7f90500af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.434+0000 7f9070200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] conn(0x7f90500ad530 0x7f90500af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90500a3d50 msgr2=0x7f90500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f90500a3d50 0x7f90500a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f906810b4a0 tx=0x7f9054009260 comp rx=0 tx=0).stop 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 msgr2=0x7f906819b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:48.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 --2- 172.21.15.19:0/1863537325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9068104ec0 0x7f906819b2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e9c0 tx=0x7f905c00ecd0 comp rx=0 tx=0).stop 2024-04-25T15:02:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f906e200700 1 -- 172.21.15.19:0/1863537325 reap_dead start 2024-04-25T15:02:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 shutdown_connections 2024-04-25T15:02:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.435+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 >> 172.21.15.19:0/1863537325 conn(0x7f9068100750 msgr2=0x7f9068101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.436+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 shutdown_connections 2024-04-25T15:02:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:47.436+0000 7f9070200700 1 -- 172.21.15.19:0/1863537325 wait complete. 2024-04-25T15:02:49.050 INFO:teuthology.orchestra.run.smithi019.stdout:111669149763 2024-04-25T15:02:49.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T15:02:49.559 INFO:teuthology.orchestra.run.smithi019.stdout:322122547229 2024-04-25T15:02:49.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T15:02:49.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:49 smithi019 ceph-mon[28090]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:49.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:49 smithi083 ceph-mon[32919]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:49.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:49 smithi102 ceph-mon[33005]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:50.282 INFO:teuthology.orchestra.run.smithi019.stdout:55834574929 2024-04-25T15:02:50.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T15:02:51.086 INFO:teuthology.orchestra.run.smithi019.stdout:167503724601 2024-04-25T15:02:51.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T15:02:51.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 -- 172.21.15.19:0/4042831080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f001810f440 msgr2=0x7f0018111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 --2- 172.21.15.19:0/4042831080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f001810f440 0x7f0018111830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f001000b0a0 tx=0x7f001000b3b0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 -- 172.21.15.19:0/4042831080 shutdown_connections 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 --2- 172.21.15.19:0/4042831080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f001810f440 0x7f0018111830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 --2- 172.21.15.19:0/4042831080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0018074c70 0x7f0018075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.380+0000 7f001e200700 1 --2- 172.21.15.19:0/4042831080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018075590 0x7f001810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.381+0000 7f001e200700 1 -- 172.21.15.19:0/4042831080 >> 172.21.15.19:0/4042831080 conn(0x7f0018079bf0 msgr2=0x7f001807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.381+0000 7f001e200700 1 -- 172.21.15.19:0/4042831080 shutdown_connections 2024-04-25T15:02:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.381+0000 7f001e200700 1 -- 172.21.15.19:0/4042831080 wait complete. 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 Processor -- start 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 -- start start 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018074c70 0x7f001819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0018075590 0x7f001819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001810f440 0x7f001819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0018114ed0 con 0x7f0018074c70 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0018114d50 con 0x7f0018075590 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.382+0000 7f001e200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0018115050 con 0x7f001810f440 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0017400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018074c70 0x7f001819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0018075590 0x7f001819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0017400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018074c70 0x7f001819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38672/0 (socket says 172.21.15.19:38672) 2024-04-25T15:02:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0017400700 1 -- 172.21.15.19:0/2732208040 learned_addr learned my addr 172.21.15.19:0/2732208040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0017e00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001810f440 0x7f001819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 -- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001810f440 msgr2=0x7f001819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001810f440 0x7f001819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 -- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018074c70 msgr2=0x7f001819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0018074c70 0x7f001819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.383+0000 7f0016a00700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00181a0390 con 0x7f0018075590 2024-04-25T15:02:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f0016a00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0018075590 0x7f001819b8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f000400e9f0 tx=0x7f000400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000400c9c0 con 0x7f0018075590 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000400cb60 con 0x7f0018075590 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00181a0680 con 0x7f0018075590 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004018680 con 0x7f0018075590 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.384+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181a7f00 con 0x7f0018075590 2024-04-25T15:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.386+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0018063190 con 0x7f0018075590 2024-04-25T15:02:51.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.389+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0004010430 con 0x7f0018075590 2024-04-25T15:02:51.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f000fe00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7efffc0a3d50 0x7efffc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f00040d0510 con 0x7f0018075590 2024-04-25T15:02:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f0017400700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7efffc0a3d50 0x7efffc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f000fe00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] conn(0x7efffc0ad530 0x7efffc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0004015e80 con 0x7efffc0ad530 2024-04-25T15:02:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.391+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f00040d7050 con 0x7f0018075590 2024-04-25T15:02:51.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.392+0000 7f0017e00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] conn(0x7efffc0ad530 0x7efffc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:51.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.392+0000 7f0017400700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7efffc0a3d50 0x7efffc0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0008009d60 tx=0x7f0008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:51.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.392+0000 7f0017e00700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] conn(0x7efffc0ad530 0x7efffc0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:51.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.393+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== osd.2 v2:172.21.15.102:6800/379851586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0004015e80 con 0x7efffc0ad530 2024-04-25T15:02:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.414+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 --> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00180635c0 con 0x7efffc0ad530 2024-04-25T15:02:51.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.415+0000 7f000fe00700 1 -- 172.21.15.19:0/2732208040 <== osd.2 v2:172.21.15.102:6800/379851586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f00180635c0 con 0x7efffc0ad530 2024-04-25T15:02:51.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.415+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] conn(0x7efffc0ad530 msgr2=0x7efffc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.415+0000 7f001e200700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] conn(0x7efffc0ad530 0x7efffc0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.415+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7efffc0a3d50 msgr2=0x7efffc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.415+0000 7f001e200700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7efffc0a3d50 0x7efffc0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0008009d60 tx=0x7f0008009260 comp rx=0 tx=0).stop 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0018075590 msgr2=0x7f001819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 --2- 172.21.15.19:0/2732208040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0018075590 0x7f001819b8c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f000400e9f0 tx=0x7f000400ed00 comp rx=0 tx=0).stop 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f0017e00700 1 -- 172.21.15.19:0/2732208040 reap_dead start 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 shutdown_connections 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 >> 172.21.15.19:0/2732208040 conn(0x7f0018079bf0 msgr2=0x7f00180ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 shutdown_connections 2024-04-25T15:02:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.416+0000 7f001e200700 1 -- 172.21.15.19:0/2732208040 wait complete. 2024-04-25T15:02:51.617 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:51 smithi019 ceph-mon[28090]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:51.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:51 smithi083 ceph-mon[32919]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:51 smithi102 ceph-mon[33005]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:52.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 -- 172.21.15.19:0/1108498627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 msgr2=0x7fa5ac107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 --2- 172.21.15.19:0/1108498627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac107d20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0099c0 tx=0x7fa59c009cd0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 -- 172.21.15.19:0/1108498627 shutdown_connections 2024-04-25T15:02:52.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 --2- 172.21.15.19:0/1108498627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac1034b0 0x7fa5ac10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 --2- 172.21.15.19:0/1108498627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac107d20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 --2- 172.21.15.19:0/1108498627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5ac102af0 0x7fa5ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 -- 172.21.15.19:0/1108498627 >> 172.21.15.19:0/1108498627 conn(0x7fa5ac079ba0 msgr2=0x7fa5ac079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.306+0000 7fa5b2400700 1 -- 172.21.15.19:0/1108498627 shutdown_connections 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.307+0000 7fa5b2400700 1 -- 172.21.15.19:0/1108498627 wait complete. 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 Processor -- start 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 -- start start 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 0x7fa5ac199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5ac1034b0 0x7fa5ac199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112c90 con 0x7fa5ac107940 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112b10 con 0x7fa5ac102af0 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.308+0000 7fa5b2400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112e10 con 0x7fa5ac1034b0 2024-04-25T15:02:52.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5ab400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 0x7fa5ac199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5ab400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 0x7fa5ac199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:54390/0 (socket says 172.21.15.19:54390) 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38720/0 (socket says 172.21.15.19:38720) 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5ab400700 1 -- 172.21.15.19:0/825101384 learned_addr learned my addr 172.21.15.19:0/825101384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5a3e00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5ac1034b0 0x7fa5ac199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 -- 172.21.15.19:0/825101384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5ac1034b0 msgr2=0x7fa5ac199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5ac1034b0 0x7fa5ac199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 -- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 msgr2=0x7fa5ac199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 0x7fa5ac199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac19e150 con 0x7fa5ac107940 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5ab400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5ac102af0 0x7fa5ac199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.309+0000 7fa5abe00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac19da10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400efc0 tx=0x7fa5a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.310+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40092e0 con 0x7fa5ac107940 2024-04-25T15:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.310+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac19e3e0 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.310+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a4004bb0 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.310+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4004020 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.310+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1a5cc0 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.311+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5ac0631d0 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.313+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa5a401b030 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.314+0000 7fa5a8c00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5900a3cd0 0x7fa5900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.315+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa5a40d0840 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.315+0000 7fa5ab400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5900a3cd0 0x7fa5900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.315+0000 7fa5a8c00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] conn(0x7fa5900ad4b0 0x7fa5900af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.315+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5900aff80 con 0x7fa5900ad4b0 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.315+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa5a40d0bf0 con 0x7fa5ac107940 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.316+0000 7fa5a3e00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] conn(0x7fa5900ad4b0 0x7fa5900af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.316+0000 7fa5ab400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5900a3cd0 0x7fa5900a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009990 tx=0x7fa59c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.316+0000 7fa5a3e00700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] conn(0x7fa5900ad4b0 0x7fa5900af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.317+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== osd.7 v2:172.21.15.83:6816/1101145206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5900aff80 con 0x7fa5900ad4b0 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.338+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 --> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5ac063650 con 0x7fa5900ad4b0 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.339+0000 7fa5a8c00700 1 -- 172.21.15.19:0/825101384 <== osd.7 v2:172.21.15.83:6816/1101145206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5ac063650 con 0x7fa5900ad4b0 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] conn(0x7fa5900ad4b0 msgr2=0x7fa5900af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] conn(0x7fa5900ad4b0 0x7fa5900af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5900a3cd0 msgr2=0x7fa5900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa5900a3cd0 0x7fa5900a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009990 tx=0x7fa59c005ab0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 msgr2=0x7fa5ac19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 --2- 172.21.15.19:0/825101384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5ac107940 0x7fa5ac19da10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400efc0 tx=0x7fa5a400c3a0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5abe00700 1 -- 172.21.15.19:0/825101384 reap_dead start 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.340+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 shutdown_connections 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.341+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 >> 172.21.15.19:0/825101384 conn(0x7fa5ac079ba0 msgr2=0x7fa5ac100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.341+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 shutdown_connections 2024-04-25T15:02:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:52.341+0000 7fa5b2400700 1 -- 172.21.15.19:0/825101384 wait complete. 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 -- 172.21.15.19:0/543274240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 msgr2=0x7fd614108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 --2- 172.21.15.19:0/543274240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 0x7fd614108e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd60400b0a0 tx=0x7fd60400b3b0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 -- 172.21.15.19:0/543274240 shutdown_connections 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 --2- 172.21.15.19:0/543274240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406b0f0 0x7fd61410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 --2- 172.21.15.19:0/543274240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 0x7fd614108e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 --2- 172.21.15.19:0/543274240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd61406a750 0x7fd61406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 -- 172.21.15.19:0/543274240 >> 172.21.15.19:0/543274240 conn(0x7fd6140788f0 msgr2=0x7fd614078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 -- 172.21.15.19:0/543274240 shutdown_connections 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.616+0000 7fd61a200700 1 -- 172.21.15.19:0/543274240 wait complete. 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 Processor -- start 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 -- start start 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 0x7fd614101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd61406b0f0 0x7fd614101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 0x7fd614102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd614112c00 con 0x7fd614108a30 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd614112a80 con 0x7fd61406b0f0 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.618+0000 7fd61a200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd614112d80 con 0x7fd61406a750 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 0x7fd614101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 0x7fd614102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 0x7fd614101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:43158/0 (socket says 172.21.15.19:43158) 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 -- 172.21.15.19:0/762386576 learned_addr learned my addr 172.21.15.19:0/762386576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd612a00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd61406b0f0 0x7fd614101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 -- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd61406b0f0 msgr2=0x7fd614101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd61406b0f0 0x7fd614101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 -- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 msgr2=0x7fd614102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd614108a30 0x7fd614102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6141a76e0 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.619+0000 7fd613400700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 0x7fd614101a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6040062a0 tx=0x7fd604004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.620+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604021070 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.620+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141a7970 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.620+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd604003c00 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.620+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604004690 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.620+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a7e20 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.621+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd614063150 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.623+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd604007a00 con 0x7fd61406a750 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.624+0000 7fd60be00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.624+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd60400f020 con 0x7fd61406a750 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd612a00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd60be00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] conn(0x7fd5f80ad380 0x7fd5f80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5f80afe50 con 0x7fd5f80ad380 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd6040d1f70 con 0x7fd61406a750 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd613e00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] conn(0x7fd5f80ad380 0x7fd5f80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.625+0000 7fd612a00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd614078ff0 tx=0x7fd60000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.626+0000 7fd613e00700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] conn(0x7fd5f80ad380 0x7fd5f80af7a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.627+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== osd.4 v2:172.21.15.83:6808/683494368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5f80afe50 con 0x7fd5f80ad380 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.645+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 --> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd614063550 con 0x7fd5f80ad380 2024-04-25T15:02:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.646+0000 7fd60be00700 1 -- 172.21.15.19:0/762386576 <== osd.4 v2:172.21.15.83:6808/683494368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd614063550 con 0x7fd5f80ad380 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.646+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] conn(0x7fd5f80ad380 msgr2=0x7fd5f80af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.646+0000 7fd61a200700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] conn(0x7fd5f80ad380 0x7fd5f80af7a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.646+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd5f80a3cd0 msgr2=0x7fd5f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd614078ff0 tx=0x7fd60000a380 comp rx=0 tx=0).stop 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 msgr2=0x7fd614101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 --2- 172.21.15.19:0/762386576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd61406a750 0x7fd614101a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6040062a0 tx=0x7fd604004e60 comp rx=0 tx=0).stop 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd613e00700 1 -- 172.21.15.19:0/762386576 reap_dead start 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 shutdown_connections 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 >> 172.21.15.19:0/762386576 conn(0x7fd6140788f0 msgr2=0x7fd6140ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 shutdown_connections 2024-04-25T15:02:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:51.647+0000 7fd61a200700 1 -- 172.21.15.19:0/762386576 wait complete. 2024-04-25T15:02:53.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 -- 172.21.15.19:0/2644857419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 msgr2=0x7f562410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 --2- 172.21.15.19:0/2644857419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 0x7f562410f8e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f561c009990 tx=0x7f561c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 -- 172.21.15.19:0/2644857419 shutdown_connections 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 --2- 172.21.15.19:0/2644857419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 0x7f562410f8e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 --2- 172.21.15.19:0/2644857419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5624108a80 0x7f5624108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 --2- 172.21.15.19:0/2644857419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f5624102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 -- 172.21.15.19:0/2644857419 >> 172.21.15.19:0/2644857419 conn(0x7f5624079bf0 msgr2=0x7f562407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 -- 172.21.15.19:0/2644857419 shutdown_connections 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.596+0000 7f562c400700 1 -- 172.21.15.19:0/2644857419 wait complete. 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.598+0000 7f562c400700 1 Processor -- start 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.598+0000 7f562c400700 1 -- start start 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f56241990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 0x7f5624199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5624108a80 0x7f562419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112c10 con 0x7f5624103450 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112a90 con 0x7f5624108a80 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562c400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112d90 con 0x7f5624102a90 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f5629000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 0x7f5624199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f5629a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f56241990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f5629a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f56241990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:43116/0 (socket says 172.21.15.19:43116) 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f5629a00700 1 -- 172.21.15.19:0/2171038969 learned_addr learned my addr 172.21.15.19:0/2171038969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.599+0000 7f562a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5624108a80 0x7f562419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 -- 172.21.15.19:0/2171038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5624108a80 msgr2=0x7f562419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5624108a80 0x7f562419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 -- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 msgr2=0x7f5624199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5624103450 0x7f5624199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419e0a0 con 0x7f5624102a90 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5629a00700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f56241990e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f561800ee10 tx=0x7f561800c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561800cbb0 con 0x7f5624102a90 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562419e390 con 0x7f5624102a90 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.600+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5618017070 con 0x7f5624102a90 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.601+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180046f0 con 0x7f5624102a90 2024-04-25T15:02:53.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.601+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5624074c70 con 0x7f5624102a90 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.602+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56240631d0 con 0x7f5624102a90 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.604+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5618003a00 con 0x7f5624102a90 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.605+0000 7f5622000700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f560c0a3cd0 0x7f560c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f56180d3ed0 con 0x7f5624102a90 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f5622000700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] conn(0x7f560c0ad4b0 0x7f560c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f561801aec0 con 0x7f560c0ad4b0 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f56180d64f0 con 0x7f5624102a90 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f5629000700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f560c0a3cd0 0x7f560c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.606+0000 7f562a400700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] conn(0x7f560c0ad4b0 0x7f560c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.607+0000 7f5629000700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f560c0a3cd0 0x7f560c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f562407a260 tx=0x7f5610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.607+0000 7f562a400700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] conn(0x7f560c0ad4b0 0x7f560c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:53.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.608+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== osd.6 v2:172.21.15.19:6818/3457660451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f561801aec0 con 0x7f560c0ad4b0 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.629+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 --> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5624063650 con 0x7f560c0ad4b0 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.630+0000 7f5622000700 1 -- 172.21.15.19:0/2171038969 <== osd.6 v2:172.21.15.19:6818/3457660451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5624063650 con 0x7f560c0ad4b0 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] conn(0x7f560c0ad4b0 msgr2=0x7f560c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] conn(0x7f560c0ad4b0 0x7f560c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f560c0a3cd0 msgr2=0x7f560c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f560c0a3cd0 0x7f560c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f562407a260 tx=0x7f5610009260 comp rx=0 tx=0).stop 2024-04-25T15:02:53.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 msgr2=0x7f56241990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 --2- 172.21.15.19:0/2171038969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5624102a90 0x7f56241990e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f561800ee10 tx=0x7f561800c650 comp rx=0 tx=0).stop 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562a400700 1 -- 172.21.15.19:0/2171038969 reap_dead start 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 shutdown_connections 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 >> 172.21.15.19:0/2171038969 conn(0x7f5624079bf0 msgr2=0x7f56241016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 shutdown_connections 2024-04-25T15:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:50.631+0000 7f562c400700 1 -- 172.21.15.19:0/2171038969 wait complete. 2024-04-25T15:02:53.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:53 smithi019 ceph-mon[28090]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:53.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:53 smithi083 ceph-mon[32919]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:53.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:53 smithi102 ceph-mon[33005]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:54.265 INFO:teuthology.orchestra.run.smithi019.stdout:146028888129 2024-04-25T15:02:54.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T15:02:54.391 INFO:teuthology.orchestra.run.smithi019.stdout:77309411404 2024-04-25T15:02:54.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T15:02:54.875 INFO:teuthology.orchestra.run.smithi019.stdout:227633266737 2024-04-25T15:02:54.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T15:02:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 -- 172.21.15.19:0/4289334222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 msgr2=0x7f7d4410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/4289334222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d4410c9f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009990 tx=0x7f7d30009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 -- 172.21.15.19:0/4289334222 shutdown_connections 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/4289334222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4410cfc0 0x7f7d4410f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/4289334222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d44101040 0x7f7d44101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/4289334222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d4410c9f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.998+0000 7f7d4aa00700 1 -- 172.21.15.19:0/4289334222 >> 172.21.15.19:0/4289334222 conn(0x7f7d440766e0 msgr2=0x7f7d44076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.999+0000 7f7d4aa00700 1 -- 172.21.15.19:0/4289334222 shutdown_connections 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:53.999+0000 7f7d4aa00700 1 -- 172.21.15.19:0/4289334222 wait complete. 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 Processor -- start 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 -- start start 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d44101040 0x7f7d441990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d441995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d4410cfc0 0x7f7d4419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44112b70 con 0x7f7d44101960 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441129f0 con 0x7f7d4410cfc0 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d4aa00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44112cf0 con 0x7f7d44101040 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.000+0000 7f7d48c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d441995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d441995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35960/0 (socket says 172.21.15.19:35960) 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 -- 172.21.15.19:0/1657104912 learned_addr learned my addr 172.21.15.19:0/1657104912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d49600700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d44101040 0x7f7d441990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d4a000700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d4410cfc0 0x7f7d4419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 -- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d44101040 msgr2=0x7f7d441990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d44101040 0x7f7d441990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 -- 172.21.15.19:0/1657104912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d4410cfc0 msgr2=0x7f7d4419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d4410cfc0 0x7f7d4419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4419e060 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.001+0000 7f7d48c00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d441995e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000c00 tx=0x7f7d300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.002+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30027070 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.002+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4419e2f0 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.002+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d30003f20 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.002+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30022c70 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.002+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441a5bd0 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.003+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d440631d0 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.005+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d3002a070 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.006+0000 7f7d42000700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7d2c0a3cd0 0x7f7d2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.006+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7d300d1d70 con 0x7f7d44101960 2024-04-25T15:02:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d49600700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7d2c0a3cd0 0x7f7d2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d42000700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] conn(0x7f7d2c0ad4b0 0x7f7d2c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d300095d0 con 0x7f7d2c0ad4b0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7d300d2120 con 0x7f7d44101960 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d4a000700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] conn(0x7f7d2c0ad4b0 0x7f7d2c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.007+0000 7f7d49600700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7d2c0a3cd0 0x7f7d2c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d38005950 tx=0x7f7d3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.008+0000 7f7d4a000700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] conn(0x7f7d2c0ad4b0 0x7f7d2c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.009+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== osd.8 v2:172.21.15.102:6816/3832909969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d300095d0 con 0x7f7d2c0ad4b0 2024-04-25T15:02:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.027+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 --> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d440635d0 con 0x7f7d2c0ad4b0 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.027+0000 7f7d42000700 1 -- 172.21.15.19:0/1657104912 <== osd.8 v2:172.21.15.102:6816/3832909969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d440635d0 con 0x7f7d2c0ad4b0 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] conn(0x7f7d2c0ad4b0 msgr2=0x7f7d2c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] conn(0x7f7d2c0ad4b0 0x7f7d2c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7d2c0a3cd0 msgr2=0x7f7d2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7d2c0a3cd0 0x7f7d2c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d38005950 tx=0x7f7d3800b410 comp rx=0 tx=0).stop 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 msgr2=0x7f7d441995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4aa00700 1 --2- 172.21.15.19:0/1657104912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d44101960 0x7f7d441995e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000c00 tx=0x7f7d300047b0 comp rx=0 tx=0).stop 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.028+0000 7f7d4a000700 1 -- 172.21.15.19:0/1657104912 reap_dead start 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.029+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 shutdown_connections 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.029+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 >> 172.21.15.19:0/1657104912 conn(0x7f7d440766e0 msgr2=0x7f7d440fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.029+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 shutdown_connections 2024-04-25T15:02:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:02:54.029+0000 7f7d4aa00700 1 -- 172.21.15.19:0/1657104912 wait complete. 2024-04-25T15:02:55.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:55 smithi019 ceph-mon[28090]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:55.769 INFO:teuthology.orchestra.run.smithi019.stdout:201863462963 2024-04-25T15:02:55.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T15:02:55.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:55 smithi083 ceph-mon[32919]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:55.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:55 smithi102 ceph-mon[33005]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:56.297 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:56.372 INFO:teuthology.orchestra.run.smithi019.stdout:266287972395 2024-04-25T15:02:56.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T15:02:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:56.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T15:02:56.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:56 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T15:02:56.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T15:02:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:56 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T15:02:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-25T15:02:56.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T15:02:56.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:56 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T15:02:57.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T15:02:57.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-25T15:02:57.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T15:02:57.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T15:02:57.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:57 smithi019 ceph-mon[28090]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T15:02:57.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:57 smithi083 ceph-mon[32919]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:57.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T15:02:57.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T15:02:57.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T15:02:57.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:57 smithi102 ceph-mon[33005]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:58.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:58 smithi019 ceph-mon[28090]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:02:58.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:58 smithi083 ceph-mon[32919]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:02:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:58 smithi102 ceph-mon[33005]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:02:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:58.865 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:59.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:02:59 smithi083 ceph-mon[32919]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:59.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:02:59 smithi102 ceph-mon[33005]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:59.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:02:59 smithi019 ceph-mon[28090]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:00 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:00 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:00.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:00 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:01.612 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:01 smithi019 ceph-mon[28090]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:01.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:01 smithi083 ceph-mon[32919]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:01.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:01 smithi102 ceph-mon[33005]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:03.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:02 smithi019 ceph-mon[28090]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:03.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.277 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:02 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:02 smithi083 ceph-mon[32919]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:02 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:02 smithi102 ceph-mon[33005]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:03.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:02 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:05.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.814+0000 7f3936600700 1 -- 172.21.15.19:0/1708539636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 msgr2=0x7f393010db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:05.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.814+0000 7f3936600700 1 --2- 172.21.15.19:0/1708539636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393010db70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f392800b0d0 tx=0x7f392800b3e0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.814+0000 7f3936600700 1 -- 172.21.15.19:0/1708539636 shutdown_connections 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 --2- 172.21.15.19:0/1708539636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393010db70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 --2- 172.21.15.19:0/1708539636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 0x7f39301012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 --2- 172.21.15.19:0/1708539636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f39301058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 -- 172.21.15.19:0/1708539636 >> 172.21.15.19:0/1708539636 conn(0x7f3930077b40 msgr2=0x7f39300feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 -- 172.21.15.19:0/1708539636 shutdown_connections 2024-04-25T15:03:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.815+0000 7f3936600700 1 -- 172.21.15.19:0/1708539636 wait complete. 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.816+0000 7f3936600700 1 Processor -- start 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.816+0000 7f3936600700 1 -- start start 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.816+0000 7f3936600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 0x7f393019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.816+0000 7f3936600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f3936600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f3936600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3930110a60 con 0x7f3930105fa0 2024-04-25T15:03:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f3936600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39301108e0 con 0x7f3930100ec0 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f3936600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3930110be0 con 0x7f3930101870 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 0x7f393019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:34148/0 (socket says 172.21.15.19:34148) 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 -- 172.21.15.19:0/1211804150 learned_addr learned my addr 172.21.15.19:0/1211804150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57892/0 (socket says 172.21.15.19:57892) 2024-04-25T15:03:05.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 -- 172.21.15.19:0/1211804150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 msgr2=0x7f393019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 0x7f393019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 -- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 msgr2=0x7f393019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.817+0000 7f392ea00700 1 -- 172.21.15.19:0/1211804150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39301a0340 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f392ea00700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ea80 tx=0x7f391c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c00ccd0 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39301a0630 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f391c004ce0 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c015390 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.818+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a7eb0 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.819+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39301023a0 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.821+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f391c004e90 con 0x7f3930101870 2024-04-25T15:03:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.823+0000 7f3927e00700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 0x7f39140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.823+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f391c0d0450 con 0x7f3930101870 2024-04-25T15:03:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.823+0000 7f392f400700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 0x7f39140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.824+0000 7f392f400700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 0x7f39140a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3920009c60 tx=0x7f3920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:04.827+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f391c09cc80 con 0x7f3930101870 2024-04-25T15:03:05.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.089+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f393019c690 con 0x7f3930101870 2024-04-25T15:03:05.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.090+0000 7f3927e00700 1 -- 172.21.15.19:0/1211804150 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f391c09c580 con 0x7f3930101870 2024-04-25T15:03:05.091 INFO:teuthology.orchestra.run.smithi019.stdout:300647710757 2024-04-25T15:03:05.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.092+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 msgr2=0x7f39140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:05.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.092+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 0x7f39140a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3920009c60 tx=0x7f3920009260 comp rx=0 tx=0).stop 2024-04-25T15:03:05.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 msgr2=0x7f393019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ea80 tx=0x7f391c00ed90 comp rx=0 tx=0).stop 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 shutdown_connections 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f39140a3cd0 0x7f39140a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3930105fa0 0x7f393019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930100ec0 0x7f393019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 --2- 172.21.15.19:0/1211804150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3930101870 0x7f393019b870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 >> 172.21.15.19:0/1211804150 conn(0x7f3930077b40 msgr2=0x7f39301046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 shutdown_connections 2024-04-25T15:03:05.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:05.093+0000 7f3936600700 1 -- 172.21.15.19:0/1211804150 wait complete. 2024-04-25T15:03:05.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:05 smithi019 ceph-mon[28090]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:05.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:05 smithi083 ceph-mon[32919]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:05.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:05 smithi102 ceph-mon[33005]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:06.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:06 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1211804150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:03:06.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:06 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1211804150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:03:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:06 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1211804150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:03:07.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:07 smithi019 ceph-mon[28090]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 -- 172.21.15.19:0/4055492259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d0806b0f0 msgr2=0x7f6d0810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 --2- 172.21.15.19:0/4055492259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d0806b0f0 0x7f6d0810f900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b0a0 tx=0x7f6d0000b3b0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 -- 172.21.15.19:0/4055492259 shutdown_connections 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 --2- 172.21.15.19:0/4055492259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d0806b0f0 0x7f6d0810f900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 --2- 172.21.15.19:0/4055492259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d08108a30 0x7f6d08108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 --2- 172.21.15.19:0/4055492259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 0x7f6d0806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 -- 172.21.15.19:0/4055492259 >> 172.21.15.19:0/4055492259 conn(0x7f6d080788f0 msgr2=0x7f6d08078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 -- 172.21.15.19:0/4055492259 shutdown_connections 2024-04-25T15:03:07.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.373+0000 7f6d0f600700 1 -- 172.21.15.19:0/4055492259 wait complete. 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.374+0000 7f6d0f600700 1 Processor -- start 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.374+0000 7f6d0f600700 1 -- start start 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 0x7f6d08199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 0x7f6d081996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112c20 con 0x7f6d08108a30 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112aa0 con 0x7f6d0806a750 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0f600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08112da0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0d600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d07e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 0x7f6d081996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0d600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57900/0 (socket says 172.21.15.19:57900) 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0d600700 1 -- 172.21.15.19:0/1236018054 learned_addr learned my addr 172.21.15.19:0/1236018054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.375+0000 7f6d0cc00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 0x7f6d08199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 -- 172.21.15.19:0/1236018054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 msgr2=0x7f6d08199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 0x7f6d08199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 -- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 msgr2=0x7f6d0819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 -- 172.21.15.19:0/1236018054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0819e150 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d0d600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d07e00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 0x7f6d081996d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9f0 tx=0x7f6cf400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf400cb70 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf400ee90 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0819e440 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4003d40 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.376+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d081a5cc0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.378+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d08069730 con 0x7f6d0806b0f0 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.379+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6cf4003ee0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.381+0000 7f6d05600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 0x7f6cf00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.381+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f6cf40d0690 con 0x7f6d0806b0f0 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.381+0000 7f6d0cc00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 0x7f6cf00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.382+0000 7f6d0cc00700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 0x7f6cf00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009be0 tx=0x7f6cfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.385+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6cf409cec0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:07 smithi083 ceph-mon[32919]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:07.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:07 smithi102 ceph-mon[33005]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.454+0000 7f69c0200700 1 -- 172.21.15.19:0/2944166245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8108a50 msgr2=0x7f69b8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.454+0000 7f69c0200700 1 --2- 172.21.15.19:0/2944166245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8108a50 0x7f69b8108e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009990 tx=0x7f69ac009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 -- 172.21.15.19:0/2944166245 shutdown_connections 2024-04-25T15:03:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 --2- 172.21.15.19:0/2944166245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8103450 0x7f69b810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 --2- 172.21.15.19:0/2944166245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8108a50 0x7f69b8108e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 --2- 172.21.15.19:0/2944166245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8102a90 0x7f69b8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 -- 172.21.15.19:0/2944166245 >> 172.21.15.19:0/2944166245 conn(0x7f69b8079ba0 msgr2=0x7f69b8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 -- 172.21.15.19:0/2944166245 shutdown_connections 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.455+0000 7f69c0200700 1 -- 172.21.15.19:0/2944166245 wait complete. 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 Processor -- start 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 -- start start 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8103450 0x7f69b819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8108a50 0x7f69b819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112c30 con 0x7f69b8108a50 2024-04-25T15:03:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112ab0 con 0x7f69b8102a90 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.456+0000 7f69c0200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112db0 con 0x7f69b8103450 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bce00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8103450 0x7f69b819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:34522/0 (socket says 172.21.15.19:34522) 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 -- 172.21.15.19:0/2274788594 learned_addr learned my addr 172.21.15.19:0/2274788594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69be200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8108a50 0x7f69b819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 -- 172.21.15.19:0/2274788594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8103450 msgr2=0x7f69b819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8103450 0x7f69b819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 -- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8108a50 msgr2=0x7f69b819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8108a50 0x7f69b819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.457+0000 7f69bd800700 1 -- 172.21.15.19:0/2274788594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b81a03b0 con 0x7f69b8102a90 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69bd800700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69ac005910 tx=0x7f69ac00b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac027070 con 0x7f69b8102a90 2024-04-25T15:03:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b81a0640 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ac022470 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac017650 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.458+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a7f20 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.459+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b8069730 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.461+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69ac02a070 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.463+0000 7f69b6000700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 0x7f69a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.463+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f69ac0d12d0 con 0x7f69b8102a90 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.463+0000 7f69bce00700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 0x7f69a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.464+0000 7f69bce00700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 0x7f69a00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69b807a4a0 tx=0x7f69a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.467+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69ac013070 con 0x7f69b8102a90 2024-04-25T15:03:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.626+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6d0819a4a0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.626+0000 7f6d05600700 1 -- 172.21.15.19:0/1236018054 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6cf409c7c0 con 0x7f6d0806b0f0 2024-04-25T15:03:07.628 INFO:teuthology.orchestra.run.smithi019.stdout:356482285594 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.629+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 msgr2=0x7f6cf00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.629+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 0x7f6cf00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009be0 tx=0x7f6cfc009260 comp rx=0 tx=0).stop 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.629+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 msgr2=0x7f6d081996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.629+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 0x7f6d081996d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9f0 tx=0x7f6cf400ed00 comp rx=0 tx=0).stop 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 shutdown_connections 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f6cf00a3cd0 0x7f6cf00a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d08108a30 0x7f6d0819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d0806a750 0x7f6d08199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 --2- 172.21.15.19:0/1236018054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d0806b0f0 0x7f6d081996d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 >> 172.21.15.19:0/1236018054 conn(0x7f6d080788f0 msgr2=0x7f6d080ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 shutdown_connections 2024-04-25T15:03:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.630+0000 7f6d0f600700 1 -- 172.21.15.19:0/1236018054 wait complete. 2024-04-25T15:03:07.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.712+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f69b802df80 con 0x7f69b8102a90 2024-04-25T15:03:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.712+0000 7f69b6000700 1 -- 172.21.15.19:0/2274788594 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f69ac09db90 con 0x7f69b8102a90 2024-04-25T15:03:07.714 INFO:teuthology.orchestra.run.smithi019.stdout:146028888132 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 msgr2=0x7f69a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 0x7f69a00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69b807a4a0 tx=0x7f69a4009260 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 msgr2=0x7f69b819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69ac005910 tx=0x7f69ac00b670 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 shutdown_connections 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f69a00a3cd0 0x7f69a00a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b8108a50 0x7f69b819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69b8102a90 0x7f69b819b3a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 --2- 172.21.15.19:0/2274788594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69b8103450 0x7f69b819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.715+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 >> 172.21.15.19:0/2274788594 conn(0x7f69b8079ba0 msgr2=0x7f69b810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:07.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.716+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 shutdown_connections 2024-04-25T15:03:07.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:07.716+0000 7f69c0200700 1 -- 172.21.15.19:0/2274788594 wait complete. 2024-04-25T15:03:08.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:08 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1236018054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:03:08.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:08 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2274788594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:03:08.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710753 got 300647710757 for osd.9 2024-04-25T15:03:08.273 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:08.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:08 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1236018054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:03:08.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:08 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2274788594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:03:08.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:08 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1236018054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:03:08.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:08 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2274788594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:03:09.284 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:09 smithi019 ceph-mon[28090]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 872 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:09.459 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:09 smithi102 ceph-mon[33005]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 872 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:09.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:09 smithi083 ceph-mon[32919]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 872 B/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T15:03:11.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:11 smithi019 ceph-mon[28090]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T15:03:11.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:11 smithi083 ceph-mon[32919]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T15:03:11.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:11 smithi102 ceph-mon[33005]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T15:03:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 -- 172.21.15.19:0/3914370894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c102a90 msgr2=0x7ff13c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 --2- 172.21.15.19:0/3914370894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c102a90 0x7ff13c102f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff128009990 tx=0x7ff128009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 -- 172.21.15.19:0/3914370894 shutdown_connections 2024-04-25T15:03:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 --2- 172.21.15.19:0/3914370894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c103450 0x7ff13c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 --2- 172.21.15.19:0/3914370894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 --2- 172.21.15.19:0/3914370894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c102a90 0x7ff13c102f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 -- 172.21.15.19:0/3914370894 >> 172.21.15.19:0/3914370894 conn(0x7ff13c079bf0 msgr2=0x7ff13c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 -- 172.21.15.19:0/3914370894 shutdown_connections 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.880+0000 7ff143600700 1 -- 172.21.15.19:0/3914370894 wait complete. 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.882+0000 7ff143600700 1 Processor -- start 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 -- start start 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c102a90 0x7ff13c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112c60 con 0x7ff13c108a80 2024-04-25T15:03:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112ae0 con 0x7ff13c103450 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.883+0000 7ff143600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112de0 con 0x7ff13c102a90 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:34532/0 (socket says 172.21.15.19:34532) 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 -- 172.21.15.19:0/2663033729 learned_addr learned my addr 172.21.15.19:0/2663033729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff140c00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c102a90 0x7ff13c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff141600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 -- 172.21.15.19:0/2663033729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c102a90 msgr2=0x7ff13c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c102a90 0x7ff13c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 -- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 msgr2=0x7ff13c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 -- 172.21.15.19:0/2663033729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c1a76e0 con 0x7ff13c103450 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff141600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.884+0000 7ff13be00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff128005910 tx=0x7ff12800b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.885+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128027070 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.885+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff128022470 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.885+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c1a7970 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.885+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128017650 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.885+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c1a7ea0 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.886+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c069730 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.888+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff12802a070 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.890+0000 7ff139600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 0x7ff1240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.890+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff1280d1850 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.890+0000 7ff140c00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 0x7ff1240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.891+0000 7ff140c00700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 0x7ff1240a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff130007b60 tx=0x7ff1300073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:11.892+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff128031070 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.136+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff13c02df80 con 0x7ff13c103450 2024-04-25T15:03:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.137+0000 7ff139600700 1 -- 172.21.15.19:0/2663033729 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff128013070 con 0x7ff13c103450 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:34359738459 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 msgr2=0x7ff1240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 0x7ff1240a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff130007b60 tx=0x7ff1300073d0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 msgr2=0x7ff13c075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff128005910 tx=0x7ff12800b670 comp rx=0 tx=0).stop 2024-04-25T15:03:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 shutdown_connections 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff1240a3d50 0x7ff1240a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff13c108a80 0x7ff13c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff13c102a90 0x7ff13c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 --2- 172.21.15.19:0/2663033729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff13c103450 0x7ff13c075280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 >> 172.21.15.19:0/2663033729 conn(0x7ff13c079bf0 msgr2=0x7ff13c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.140+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 shutdown_connections 2024-04-25T15:03:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:12.141+0000 7ff143600700 1 -- 172.21.15.19:0/2663033729 wait complete. 2024-04-25T15:03:13.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:13 smithi019 ceph-mon[28090]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:13.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:13 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2663033729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:03:13.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:13 smithi083 ceph-mon[32919]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:13.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:13 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2663033729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:03:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:13 smithi102 ceph-mon[33005]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:13.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:13 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2663033729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:03:14.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888129 got 146028888132 for osd.4 2024-04-25T15:03:14.051 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:14.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285589 got 356482285594 for osd.11 2024-04-25T15:03:14.492 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.905+0000 7f9924c00700 1 -- 172.21.15.19:0/1942158046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9920108a30 msgr2=0x7f9920108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.905+0000 7f9924c00700 1 --2- 172.21.15.19:0/1942158046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9920108a30 0x7f9920108e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99100098f0 tx=0x7f9910009c00 comp rx=0 tx=0).stop 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 -- 172.21.15.19:0/1942158046 shutdown_connections 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 --2- 172.21.15.19:0/1942158046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f992006b0f0 0x7f992010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 --2- 172.21.15.19:0/1942158046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9920108a30 0x7f9920108e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 --2- 172.21.15.19:0/1942158046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 -- 172.21.15.19:0/1942158046 >> 172.21.15.19:0/1942158046 conn(0x7f99200788f0 msgr2=0x7f9920078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 -- 172.21.15.19:0/1942158046 shutdown_connections 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.906+0000 7f9924c00700 1 -- 172.21.15.19:0/1942158046 wait complete. 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 Processor -- start 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 -- start start 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f992006b0f0 0x7f992019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920108a30 0x7f992019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112c00 con 0x7f9920108a30 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112a80 con 0x7f992006a750 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.907+0000 7f9924c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112d80 con 0x7f992006b0f0 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920108a30 0x7f992019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40152/0 (socket says 172.21.15.19:40152) 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 -- 172.21.15.19:0/1744334203 learned_addr learned my addr 172.21.15.19:0/1744334203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f9917e00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f992006b0f0 0x7f992019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 -- 172.21.15.19:0/1744334203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f992006b0f0 msgr2=0x7f992019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f992006b0f0 0x7f992019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 -- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920108a30 msgr2=0x7f992019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920108a30 0x7f992019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 -- 172.21.15.19:0/1744334203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99201a03a0 con 0x7f992006a750 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.908+0000 7f991de00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9910009230 tx=0x7f991000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.909+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100050c0 con 0x7f992006a750 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.909+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201a0630 con 0x7f992006a750 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.909+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991000b560 con 0x7f992006a750 2024-04-25T15:03:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.909+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910003a00 con 0x7f992006a750 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.909+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a7f10 con 0x7f992006a750 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.910+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9920069730 con 0x7f992006a750 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.912+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f991000b710 con 0x7f992006a750 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.913+0000 7f9916a00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 0x7f99040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.913+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f99100d1640 con 0x7f992006a750 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.914+0000 7f9917e00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 0x7f99040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.914+0000 7f9917e00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 0x7f99040a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9908009680 tx=0x7f9908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:14.915+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f991005eff0 con 0x7f992006a750 2024-04-25T15:03:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.156+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f992019c6f0 con 0x7f992006a750 2024-04-25T15:03:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.157+0000 7f9916a00700 1 -- 172.21.15.19:0/1744334203 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f992019c6f0 con 0x7f992006a750 2024-04-25T15:03:15.159 INFO:teuthology.orchestra.run.smithi019.stdout:55834574935 2024-04-25T15:03:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 msgr2=0x7f99040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 0x7f99040a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9908009680 tx=0x7f9908009260 comp rx=0 tx=0).stop 2024-04-25T15:03:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 msgr2=0x7f992019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9910009230 tx=0x7f991000fa70 comp rx=0 tx=0).stop 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 shutdown_connections 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f99040a3d50 0x7f99040a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920108a30 0x7f992019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992006a750 0x7f992019b390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 --2- 172.21.15.19:0/1744334203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f992006b0f0 0x7f992019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 >> 172.21.15.19:0/1744334203 conn(0x7f99200788f0 msgr2=0x7f99200ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 shutdown_connections 2024-04-25T15:03:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.160+0000 7f9924c00700 1 -- 172.21.15.19:0/1744334203 wait complete. 2024-04-25T15:03:15.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:15 smithi019 ceph-mon[28090]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:15.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:15 smithi083 ceph-mon[32919]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:15.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:15 smithi102 ceph-mon[33005]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:16.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 -- 172.21.15.19:0/446555531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8910109c80 msgr2=0x7f891010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 --2- 172.21.15.19:0/446555531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8910109c80 0x7f891010dcd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0099c0 tx=0x7f88fc009cd0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 -- 172.21.15.19:0/446555531 shutdown_connections 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 --2- 172.21.15.19:0/446555531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f891010e3b0 0x7f8910115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 --2- 172.21.15.19:0/446555531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89101092d0 0x7f89101096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 --2- 172.21.15.19:0/446555531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8910109c80 0x7f891010dcd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 -- 172.21.15.19:0/446555531 >> 172.21.15.19:0/446555531 conn(0x7f8910076210 msgr2=0x7f8910107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 -- 172.21.15.19:0/446555531 shutdown_connections 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.990+0000 7f8916e00700 1 -- 172.21.15.19:0/446555531 wait complete. 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.991+0000 7f8916e00700 1 Processor -- start 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 -- start start 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8910109c80 0x7f891019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f891010e3b0 0x7f89101a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8910119220 con 0x7f8910109c80 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89101190a0 con 0x7f89101092d0 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8916e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f89101193a0 con 0x7f891010e3b0 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f890fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f890f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8910109c80 0x7f891019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f890fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40170/0 (socket says 172.21.15.19:40170) 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f890fe00700 1 -- 172.21.15.19:0/694015022 learned_addr learned my addr 172.21.15.19:0/694015022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.992+0000 7f8914e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f891010e3b0 0x7f89101a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 -- 172.21.15.19:0/694015022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f891010e3b0 msgr2=0x7f89101a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f891010e3b0 0x7f89101a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 -- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8910109c80 msgr2=0x7f891019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8910109c80 0x7f891019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 -- 172.21.15.19:0/694015022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89101a4720 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890fe00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f890000eb30 tx=0x7f890000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000cbd0 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.993+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89101a4a10 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.994+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89000044d0 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.994+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900003df0 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.994+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101ac290 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.995+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910069730 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.997+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8900004000 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.999+0000 7f890cc00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 0x7f88f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.999+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f89000d0870 con 0x7f89101092d0 2024-04-25T15:03:16.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:15.999+0000 7f890f400700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 0x7f88f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.000+0000 7f890f400700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 0x7f88f40a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000c00 tx=0x7f88fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:16.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.000+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89000042f0 con 0x7f89101092d0 2024-04-25T15:03:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.239+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f89101a0a70 con 0x7f89101092d0 2024-04-25T15:03:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.239+0000 7f890cc00700 1 -- 172.21.15.19:0/694015022 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f890009d190 con 0x7f89101092d0 2024-04-25T15:03:16.241 INFO:teuthology.orchestra.run.smithi019.stdout:322122547235 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 msgr2=0x7f88f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 0x7f88f40a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000c00 tx=0x7f88fc005fb0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 msgr2=0x7f891019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f890000eb30 tx=0x7f890000ee40 comp rx=0 tx=0).stop 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 shutdown_connections 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f891010e3b0 0x7f89101a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89101092d0 0x7f891019f710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f88f40a3d50 0x7f88f40a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 --2- 172.21.15.19:0/694015022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8910109c80 0x7f891019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 >> 172.21.15.19:0/694015022 conn(0x7f8910076210 msgr2=0x7f8910077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 shutdown_connections 2024-04-25T15:03:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.243+0000 7f8916e00700 1 -- 172.21.15.19:0/694015022 wait complete. 2024-04-25T15:03:16.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1744334203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:03:16.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1744334203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:03:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1744334203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:03:16.638 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738459 for osd.0 2024-04-25T15:03:16.639 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 -- 172.21.15.19:0/3983510636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e8109c80 msgr2=0x7f77e810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 --2- 172.21.15.19:0/3983510636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e8109c80 0x7f77e810dcd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009990 tx=0x7f77d4009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 -- 172.21.15.19:0/3983510636 shutdown_connections 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 --2- 172.21.15.19:0/3983510636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e810e3b0 0x7f77e8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 --2- 172.21.15.19:0/3983510636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e81092d0 0x7f77e81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 --2- 172.21.15.19:0/3983510636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e8109c80 0x7f77e810dcd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 -- 172.21.15.19:0/3983510636 >> 172.21.15.19:0/3983510636 conn(0x7f77e8076210 msgr2=0x7f77e8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 -- 172.21.15.19:0/3983510636 shutdown_connections 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.425+0000 7f77efa00700 1 -- 172.21.15.19:0/3983510636 wait complete. 2024-04-25T15:03:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 Processor -- start 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 -- start start 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e81092d0 0x7f77e81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e810e3b0 0x7f77e819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8119270 con 0x7f77e81092d0 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81190f0 con 0x7f77e810e3b0 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.427+0000 7f77efa00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81193f0 con 0x7f77e8109c80 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:54142/0 (socket says 172.21.15.19:54142) 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 -- 172.21.15.19:0/2203465342 learned_addr learned my addr 172.21.15.19:0/2203465342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77ed000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e81092d0 0x7f77e81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77eda00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e810e3b0 0x7f77e819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 -- 172.21.15.19:0/2203465342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e810e3b0 msgr2=0x7f77e819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e810e3b0 0x7f77e819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 -- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e81092d0 msgr2=0x7f77e81a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e81092d0 0x7f77e81a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.428+0000 7f77e7e00700 1 -- 172.21.15.19:0/2203465342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e819fd50 con 0x7f77e8109c80 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77e7e00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000c00 tx=0x7f77d40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4027070 con 0x7f77e8109c80 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d400b610 con 0x7f77e8109c80 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40173f0 con 0x7f77e8109c80 2024-04-25T15:03:16.864 INFO:teuthology.orchestra.run.smithi019.stdout:111669149770 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e819ffe0 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.429+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81ac300 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.430+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8069730 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.432+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77d402a070 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.433+0000 7f77e5600700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 0x7f77d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.434+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f77d40d1470 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.434+0000 7f77ed000700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 0x7f77d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.434+0000 7f77ed000700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 0x7f77d00a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009c60 tx=0x7f77dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.436+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77d4013070 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.674+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f77e802df80 con 0x7f77e8109c80 2024-04-25T15:03:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.675+0000 7f77e5600700 1 -- 172.21.15.19:0/2203465342 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f77d409dd30 con 0x7f77e8109c80 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 msgr2=0x7f77d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 0x7f77d00a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009c60 tx=0x7f77dc009260 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 msgr2=0x7f77e81a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000c00 tx=0x7f77d40057c0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 shutdown_connections 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77e810e3b0 0x7f77e819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f77d00a3d50 0x7f77d00a6210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e81092d0 0x7f77e81a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 --2- 172.21.15.19:0/2203465342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77e8109c80 0x7f77e81a5970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 >> 172.21.15.19:0/2203465342 conn(0x7f77e8076210 msgr2=0x7f77e8077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 shutdown_connections 2024-04-25T15:03:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:16.678+0000 7f77efa00700 1 -- 172.21.15.19:0/2203465342 wait complete. 2024-04-25T15:03:17.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:17 smithi019 ceph-mon[28090]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:17.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:17 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/694015022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:03:17.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:17 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2203465342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:03:17.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:17 smithi083 ceph-mon[32919]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:17.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:17 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/694015022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:03:17.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:17 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2203465342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:03:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:17 smithi102 ceph-mon[33005]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:17 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/694015022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:03:17.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:17 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2203465342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:03:18.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149763 got 111669149770 for osd.3 2024-04-25T15:03:18.133 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:18.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574935 for osd.1 2024-04-25T15:03:18.285 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 -- 172.21.15.19:0/3330785657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca0107940 msgr2=0x7fbca0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 --2- 172.21.15.19:0/3330785657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca0107940 0x7fbca0107d20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000fb20 tx=0x7fbc9000fe30 comp rx=0 tx=0).stop 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 -- 172.21.15.19:0/3330785657 shutdown_connections 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 --2- 172.21.15.19:0/3330785657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca01034b0 0x7fbca010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 --2- 172.21.15.19:0/3330785657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca0107940 0x7fbca0107d20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 --2- 172.21.15.19:0/3330785657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0102af0 0x7fbca0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 -- 172.21.15.19:0/3330785657 >> 172.21.15.19:0/3330785657 conn(0x7fbca0079ba0 msgr2=0x7fbca0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 -- 172.21.15.19:0/3330785657 shutdown_connections 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.496+0000 7fbca6400700 1 -- 172.21.15.19:0/3330785657 wait complete. 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.497+0000 7fbca6400700 1 Processor -- start 2024-04-25T15:03:18.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.498+0000 7fbca6400700 1 -- start start 2024-04-25T15:03:18.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.498+0000 7fbca6400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:18.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.498+0000 7fbca6400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 0x7fbca0074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:18.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.499+0000 7fbca6400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0107940 0x7fbca006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:18.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.499+0000 7fbc9f400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.499+0000 7fbc9f400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:54158/0 (socket says 172.21.15.19:54158) 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbca6400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0121b00 con 0x7fbca01034b0 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbca6400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0121980 con 0x7fbca0107940 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbca6400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0121c80 con 0x7fbca0102af0 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 -- 172.21.15.19:0/2691364956 learned_addr learned my addr 172.21.15.19:0/2691364956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9fe00700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0107940 0x7fbca006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 -- 172.21.15.19:0/2691364956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0107940 msgr2=0x7fbca006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:18.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0107940 0x7fbca006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 -- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 msgr2=0x7fbca0074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.499+0000 7fbc96a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 0x7fbca0074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 0x7fbca0074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc9f400700 1 -- 172.21.15.19:0/2691364956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca006dd40 con 0x7fbca0102af0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.500+0000 7fbc96a00700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 0x7fbca0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.501+0000 7fbc9f400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009db0 tx=0x7fbc90009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.501+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003a90 con 0x7fbca0102af0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.501+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc90013030 con 0x7fbca0102af0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.501+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000d3e0 con 0x7fbca0102af0 2024-04-25T15:03:18.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.501+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca006dfd0 con 0x7fbca0102af0 2024-04-25T15:03:18.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.502+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca0071f30 con 0x7fbca0102af0 2024-04-25T15:03:18.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.503+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca01041b0 con 0x7fbca0102af0 2024-04-25T15:03:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.505+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc90019070 con 0x7fbca0102af0 2024-04-25T15:03:18.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.506+0000 7fbc9cc00700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 0x7fbc840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.506+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbc900d0c30 con 0x7fbca0102af0 2024-04-25T15:03:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.507+0000 7fbc96a00700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 0x7fbc840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.507+0000 7fbc96a00700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 0x7fbc840a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbca006ea20 tx=0x7fbc88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.510+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc9009d600 con 0x7fbca0102af0 2024-04-25T15:03:18.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.751+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbca0072960 con 0x7fbca0102af0 2024-04-25T15:03:18.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.752+0000 7fbc9cc00700 1 -- 172.21.15.19:0/2691364956 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc9009cf00 con 0x7fbca0102af0 2024-04-25T15:03:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:201863462968 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.754+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 msgr2=0x7fbc840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.754+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 0x7fbc840a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbca006ea20 tx=0x7fbc88008040 comp rx=0 tx=0).stop 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 msgr2=0x7fbca00741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009db0 tx=0x7fbc90009de0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 shutdown_connections 2024-04-25T15:03:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbca0107940 0x7fbca006d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbca0102af0 0x7fbca00741d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc840a3d50 0x7fbc840a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 --2- 172.21.15.19:0/2691364956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbca01034b0 0x7fbca0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 >> 172.21.15.19:0/2691364956 conn(0x7fbca0079ba0 msgr2=0x7fbca0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 shutdown_connections 2024-04-25T15:03:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:18.755+0000 7fbca6400700 1 -- 172.21.15.19:0/2691364956 wait complete. 2024-04-25T15:03:19.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:19 smithi083 ceph-mon[32919]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:19.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2691364956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:03:19.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:19 smithi102 ceph-mon[33005]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:19.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2691364956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:03:19.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:19 smithi019 ceph-mon[28090]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:19.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2691364956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:03:19.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462968 for osd.6 2024-04-25T15:03:19.814 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 -- 172.21.15.19:0/1089619775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 msgr2=0x7f1b081052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/1089619775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b081052a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1af80099c0 tx=0x7f1af8009cd0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 -- 172.21.15.19:0/1089619775 shutdown_connections 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/1089619775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 0x7f1b08111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/1089619775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b081052a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/1089619775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 -- 172.21.15.19:0/1089619775 >> 172.21.15.19:0/1089619775 conn(0x7f1b08100750 msgr2=0x7f1b08102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 -- 172.21.15.19:0/1089619775 shutdown_connections 2024-04-25T15:03:20.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.563+0000 7f1b0ec00700 1 -- 172.21.15.19:0/1089619775 wait complete. 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.564+0000 7f1b0ec00700 1 Processor -- start 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.564+0000 7f1b0ec00700 1 -- start start 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b0819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 0x7f1b0819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08114e80 con 0x7f1b08104ec0 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08114d00 con 0x7f1b08109fa0 2024-04-25T15:03:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0ec00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08115000 con 0x7f1b08105870 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b07e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b07400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b0819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b07e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41288/0 (socket says 172.21.15.19:41288) 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b07e00700 1 -- 172.21.15.19:0/3967038110 learned_addr learned my addr 172.21.15.19:0/3967038110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.565+0000 7f1b0cc00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 0x7f1b0819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 -- 172.21.15.19:0/3967038110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 msgr2=0x7f1b0819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 0x7f1b0819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 -- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 msgr2=0x7f1b0819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 -- 172.21.15.19:0/3967038110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b081a0360 con 0x7f1b08105870 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07e00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:20.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b07400700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b0819b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e9c0 tx=0x7f1af400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af400c9c0 con 0x7f1b08105870 2024-04-25T15:03:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af400cb60 con 0x7f1b08105870 2024-04-25T15:03:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.566+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b081a0650 con 0x7f1b08105870 2024-04-25T15:03:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.567+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b081a7ed0 con 0x7f1b08105870 2024-04-25T15:03:20.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.567+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4018680 con 0x7f1b08105870 2024-04-25T15:03:20.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.568+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae8005330 con 0x7f1b08105870 2024-04-25T15:03:20.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.570+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1af4018820 con 0x7f1b08105870 2024-04-25T15:03:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.571+0000 7f1b04c00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 0x7f1aec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.571+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f1af40d06d0 con 0x7f1b08105870 2024-04-25T15:03:20.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.572+0000 7f1b07e00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 0x7f1aec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:20.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.572+0000 7f1b07e00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 0x7f1aec0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000c00 tx=0x7f1af8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:20.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.573+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1af409cf00 con 0x7f1b08105870 2024-04-25T15:03:20.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.821+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1ae80058f0 con 0x7f1b08105870 2024-04-25T15:03:20.822 INFO:teuthology.orchestra.run.smithi019.stdout:77309411410 2024-04-25T15:03:20.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.821+0000 7f1b04c00700 1 -- 172.21.15.19:0/3967038110 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1af409c800 con 0x7f1b08105870 2024-04-25T15:03:20.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.824+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 msgr2=0x7f1aec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.824+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 0x7f1aec0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000c00 tx=0x7f1af8005fd0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.824+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 msgr2=0x7f1b0819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.824+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b0819b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e9c0 tx=0x7f1af400ecd0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 shutdown_connections 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b08109fa0 0x7f1b0819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f1aec0a3d50 0x7f1aec0a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b08104ec0 0x7f1b0819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 --2- 172.21.15.19:0/3967038110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b08105870 0x7f1b0819b890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 >> 172.21.15.19:0/3967038110 conn(0x7f1b08100750 msgr2=0x7f1b08102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 shutdown_connections 2024-04-25T15:03:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:20.825+0000 7f1b0ec00700 1 -- 172.21.15.19:0/3967038110 wait complete. 2024-04-25T15:03:21.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:21 smithi019 ceph-mon[28090]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:21.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:21 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3967038110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:03:21.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:21 smithi083 ceph-mon[32919]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:21.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:21 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3967038110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:03:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:21 smithi102 ceph-mon[33005]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:21.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:21 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3967038110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 -- 172.21.15.19:0/2906467860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8101940 msgr2=0x7f00e8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 --2- 172.21.15.19:0/2906467860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8101940 0x7f00e8101dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009990 tx=0x7f00d4009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 -- 172.21.15.19:0/2906467860 shutdown_connections 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 --2- 172.21.15.19:0/2906467860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8102300 0x7f00e810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 --2- 172.21.15.19:0/2906467860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8107990 0x7f00e8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 --2- 172.21.15.19:0/2906467860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8101940 0x7f00e8101dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 -- 172.21.15.19:0/2906467860 >> 172.21.15.19:0/2906467860 conn(0x7f00e80788f0 msgr2=0x7f00e8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.642+0000 7f00f0200700 1 -- 172.21.15.19:0/2906467860 shutdown_connections 2024-04-25T15:03:21.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.643+0000 7f00f0200700 1 -- 172.21.15.19:0/2906467860 wait complete. 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.644+0000 7f00f0200700 1 Processor -- start 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 -- start start 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 0x7f00e81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 0x7f00e8103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111af0 con 0x7f00e8102300 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111970 con 0x7f00e8107990 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.645+0000 7f00f0200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111c70 con 0x7f00e8101940 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ece00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 0x7f00e8103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ed800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 0x7f00e81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ed800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 0x7f00e81034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:54222/0 (socket says 172.21.15.19:54222) 2024-04-25T15:03:21.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ece00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 0x7f00e8103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41316/0 (socket says 172.21.15.19:41316) 2024-04-25T15:03:21.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ed800700 1 -- 172.21.15.19:0/2864582004 learned_addr learned my addr 172.21.15.19:0/2864582004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:21.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40266/0 (socket says 172.21.15.19:40266) 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 -- 172.21.15.19:0/2864582004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 msgr2=0x7f00e81034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 0x7f00e81034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 -- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 msgr2=0x7f00e8103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 0x7f00e8103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 -- 172.21.15.19:0/2864582004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81a7640 con 0x7f00e8107990 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.646+0000 7f00ee200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00e000efc0 tx=0x7f00e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.647+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e00092e0 con 0x7f00e8107990 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.647+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a78d0 con 0x7f00e8107990 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.647+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e0004bb0 con 0x7f00e8107990 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.647+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0004020 con 0x7f00e8107990 2024-04-25T15:03:21.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.647+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a7de0 con 0x7f00e8107990 2024-04-25T15:03:21.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.648+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8069730 con 0x7f00e8107990 2024-04-25T15:03:21.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.650+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f00e001b030 con 0x7f00e8107990 2024-04-25T15:03:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.651+0000 7f00e6000700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.652+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f00e00cfff0 con 0x7f00e8107990 2024-04-25T15:03:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.652+0000 7f00ed800700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.652+0000 7f00ed800700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 0x7f00d00a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009ce0 tx=0x7f00dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:21.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.656+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00e0022020 con 0x7f00e8107990 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 -- 172.21.15.19:0/322158766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 msgr2=0x7f5d88107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/322158766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d88107d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b0a0 tx=0x7f5d7800b3b0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 -- 172.21.15.19:0/322158766 shutdown_connections 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/322158766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88102300 0x7f5d8810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/322158766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d88107d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/322158766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88101940 0x7f5d88101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 -- 172.21.15.19:0/322158766 >> 172.21.15.19:0/322158766 conn(0x7f5d880788f0 msgr2=0x7f5d88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 -- 172.21.15.19:0/322158766 shutdown_connections 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.550+0000 7f5d8cc00700 1 -- 172.21.15.19:0/322158766 wait complete. 2024-04-25T15:03:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.552+0000 7f5d8cc00700 1 Processor -- start 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.552+0000 7f5d8cc00700 1 -- start start 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88102300 0x7f5d88199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d8819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88111b50 con 0x7f5d88107990 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881119d0 con 0x7f5d88101940 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d8cc00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88111cd0 con 0x7f5d88102300 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40252/0 (socket says 172.21.15.19:40252) 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 -- 172.21.15.19:0/1439187014 learned_addr learned my addr 172.21.15.19:0/1439187014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d7d400700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88102300 0x7f5d88199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d86800700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d8819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 -- 172.21.15.19:0/1439187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88102300 msgr2=0x7f5d88199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88102300 0x7f5d88199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 -- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 msgr2=0x7f5d8819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d8819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.553+0000 7f5d85e00700 1 -- 172.21.15.19:0/1439187014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8819e180 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d85e00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800d010 tx=0x7f5d78004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78021070 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d78007c40 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780046a0 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8819e410 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.554+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881a5cf0 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.555+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d88103060 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.559+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d78003a00 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.560+0000 7f5d7f400700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 0x7f5d6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.561+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5d7800f020 con 0x7f5d88101940 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.561+0000 7f5d7d400700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 0x7f5d6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.561+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d7802a020 con 0x7f5d88101940 2024-04-25T15:03:21.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.562+0000 7f5d7d400700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 0x7f5d6c0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d88078ef0 tx=0x7f5d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:21.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.798+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5d881a5ec0 con 0x7f5d88101940 2024-04-25T15:03:21.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.799+0000 7f5d7f400700 1 -- 172.21.15.19:0/1439187014 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d7809e680 con 0x7f5d88101940 2024-04-25T15:03:21.800 INFO:teuthology.orchestra.run.smithi019.stdout:167503724608 2024-04-25T15:03:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 msgr2=0x7f5d6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 0x7f5d6c0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d88078ef0 tx=0x7f5d70009260 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 msgr2=0x7f5d881991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800d010 tx=0x7f5d78004d00 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 shutdown_connections 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5d6c0a3d50 0x7f5d6c0a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d88107990 0x7f5d8819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d88101940 0x7f5d881991c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 --2- 172.21.15.19:0/1439187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d88102300 0x7f5d88199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 >> 172.21.15.19:0/1439187014 conn(0x7f5d880788f0 msgr2=0x7f5d8806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 shutdown_connections 2024-04-25T15:03:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.802+0000 7f5d8cc00700 1 -- 172.21.15.19:0/1439187014 wait complete. 2024-04-25T15:03:21.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.896+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f00e802e140 con 0x7f00e8107990 2024-04-25T15:03:21.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.897+0000 7f00e6000700 1 -- 172.21.15.19:0/2864582004 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00e0010040 con 0x7f00e8107990 2024-04-25T15:03:21.898 INFO:teuthology.orchestra.run.smithi019.stdout:227633266742 2024-04-25T15:03:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 msgr2=0x7f00d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 0x7f00d00a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009ce0 tx=0x7f00dc009260 comp rx=0 tx=0).stop 2024-04-25T15:03:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 msgr2=0x7f00e8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00e000efc0 tx=0x7f00e000c3a0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 shutdown_connections 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8107990 0x7f00e8103f70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00e8101940 0x7f00e81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00d00a3d50 0x7f00d00a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 --2- 172.21.15.19:0/2864582004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8102300 0x7f00e8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 >> 172.21.15.19:0/2864582004 conn(0x7f00e80788f0 msgr2=0x7f00e806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 shutdown_connections 2024-04-25T15:03:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:21.900+0000 7f00f0200700 1 -- 172.21.15.19:0/2864582004 wait complete. 2024-04-25T15:03:22.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:22 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1439187014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:03:22.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:22 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2864582004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:03:22.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:22 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1439187014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:03:22.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:22 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2864582004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:03:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:22 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1439187014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:03:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:22 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2864582004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:03:22.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547229 got 322122547235 for osd.10 2024-04-25T15:03:22.985 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:23.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:23 smithi019 ceph-mon[28090]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:23.406 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411410 for osd.2 2024-04-25T15:03:23.407 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:23.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:23 smithi083 ceph-mon[32919]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:23.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:23 smithi102 ceph-mon[33005]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:23.581 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:24.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724608 for osd.5 2024-04-25T15:03:24.100 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:24.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266737 got 227633266742 for osd.7 2024-04-25T15:03:24.649 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:25.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:25 smithi083 ceph-mon[32919]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:25.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:25 smithi102 ceph-mon[33005]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:25.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:25 smithi019 ceph-mon[28090]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 -- 172.21.15.19:0/1694343644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 msgr2=0x7f5900108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 --2- 172.21.15.19:0/1694343644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f5900108e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009990 tx=0x7f58f0009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 -- 172.21.15.19:0/1694343644 shutdown_connections 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 --2- 172.21.15.19:0/1694343644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 0x7f590010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 --2- 172.21.15.19:0/1694343644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f5900108e30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 --2- 172.21.15.19:0/1694343644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 -- 172.21.15.19:0/1694343644 >> 172.21.15.19:0/1694343644 conn(0x7f5900079ba0 msgr2=0x7f5900079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 -- 172.21.15.19:0/1694343644 shutdown_connections 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.339+0000 7f5906200700 1 -- 172.21.15.19:0/1694343644 wait complete. 2024-04-25T15:03:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.340+0000 7f5906200700 1 Processor -- start 2024-04-25T15:03:27.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.340+0000 7f5906200700 1 -- start start 2024-04-25T15:03:27.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.340+0000 7f5906200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f5906200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 0x7f5900075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f5906200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f59000757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f5906200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112c00 con 0x7f5900108a50 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f5906200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112a80 con 0x7f5900103450 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f5906200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112d80 con 0x7f5900102a90 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:35596/0 (socket says 172.21.15.19:35596) 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 -- 172.21.15.19:0/3698113158 learned_addr learned my addr 172.21.15.19:0/3698113158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ffe00700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f59000757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58fea00700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 0x7f5900075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:27.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 -- 172.21.15.19:0/3698113158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 msgr2=0x7f5900075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 0x7f5900075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 -- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 msgr2=0x7f59000757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f59000757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.341+0000 7f58ff400700 1 -- 172.21.15.19:0/3698113158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59001a7740 con 0x7f5900102a90 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f58ff400700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58f0000c00 tx=0x7f58f000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0027070 con 0x7f5900102a90 2024-04-25T15:03:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0022470 con 0x7f5900102a90 2024-04-25T15:03:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001a79d0 con 0x7f5900102a90 2024-04-25T15:03:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0017650 con 0x7f5900102a90 2024-04-25T15:03:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.342+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a7f00 con 0x7f5900102a90 2024-04-25T15:03:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.344+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900069730 con 0x7f5900102a90 2024-04-25T15:03:27.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.345+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f58f002a070 con 0x7f5900102a90 2024-04-25T15:03:27.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.347+0000 7f58f7e00700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:27.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.347+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f58f00d0ff0 con 0x7f5900102a90 2024-04-25T15:03:27.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.348+0000 7f58fea00700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:27.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.349+0000 7f58fea00700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 0x7f58e40a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f590007a350 tx=0x7f58ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:27.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.351+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58f0013070 con 0x7f5900102a90 2024-04-25T15:03:27.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:27 smithi083 ceph-mon[32919]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:27 smithi102 ceph-mon[33005]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:27.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:27 smithi019 ceph-mon[28090]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:27.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.629+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f590002df80 con 0x7f5900102a90 2024-04-25T15:03:27.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.629+0000 7f58f7e00700 1 -- 172.21.15.19:0/3698113158 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58f009da30 con 0x7f5900102a90 2024-04-25T15:03:27.630 INFO:teuthology.orchestra.run.smithi019.stdout:266287972402 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 msgr2=0x7f58e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 0x7f58e40a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f590007a350 tx=0x7f58ec009260 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 msgr2=0x7f5900074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58f0000c00 tx=0x7f58f000bfd0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 shutdown_connections 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5900108a50 0x7f59000757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5900102a90 0x7f5900074d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 --2- 172.21.15.19:0/3698113158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5900103450 0x7f5900075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 >> 172.21.15.19:0/3698113158 conn(0x7f5900079ba0 msgr2=0x7f59001019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:27.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 shutdown_connections 2024-04-25T15:03:27.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:27.632+0000 7f5906200700 1 -- 172.21.15.19:0/3698113158 wait complete. 2024-04-25T15:03:28.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:28 smithi019 ceph-mon[28090]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:03:28.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:28 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3698113158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:03:28.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:28 smithi083 ceph-mon[32919]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:03:28.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:28 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3698113158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:03:28.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:28 smithi102 ceph-mon[33005]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:03:28.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:28 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3698113158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:03:28.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972395 got 266287972402 for osd.8 2024-04-25T15:03:28.562 DEBUG:teuthology.parallel:result is None 2024-04-25T15:03:28.562 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T15:03:28.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:03:29.040 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:29.512 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:29 smithi019 ceph-mon[28090]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:29.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:29.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:29 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:29.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:29 smithi083 ceph-mon[32919]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:29.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:29.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:29 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:29.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:29 smithi102 ceph-mon[33005]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:29.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:29.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:29 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:31.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 -- 172.21.15.19:0/510102101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806b0f0 msgr2=0x7fb1d810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 --2- 172.21.15.19:0/510102101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806b0f0 0x7fb1d810f900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b0a0 tx=0x7fb1d000b3b0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 -- 172.21.15.19:0/510102101 shutdown_connections 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 --2- 172.21.15.19:0/510102101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806b0f0 0x7fb1d810f900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 --2- 172.21.15.19:0/510102101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 0x7fb1d8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 --2- 172.21.15.19:0/510102101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806a750 0x7fb1d806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.390+0000 7fb1dd000700 1 -- 172.21.15.19:0/510102101 >> 172.21.15.19:0/510102101 conn(0x7fb1d80788f0 msgr2=0x7fb1d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.391+0000 7fb1dd000700 1 -- 172.21.15.19:0/510102101 shutdown_connections 2024-04-25T15:03:31.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.391+0000 7fb1dd000700 1 -- 172.21.15.19:0/510102101 wait complete. 2024-04-25T15:03:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.392+0000 7fb1dd000700 1 Processor -- start 2024-04-25T15:03:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.392+0000 7fb1dd000700 1 -- start start 2024-04-25T15:03:31.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806a750 0x7fb1d81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 0x7fb1d819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8112c60 con 0x7fb1d8108a30 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8112ae0 con 0x7fb1d806a750 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1dd000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8112de0 con 0x7fb1d806b0f0 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1d6800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 0x7fb1d819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1d5400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1d5e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806a750 0x7fb1d81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1d5400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:35616/0 (socket says 172.21.15.19:35616) 2024-04-25T15:03:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.393+0000 7fb1d5400700 1 -- 172.21.15.19:0/3388773182 learned_addr learned my addr 172.21.15.19:0/3388773182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 -- 172.21.15.19:0/3388773182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806a750 msgr2=0x7fb1d81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806a750 0x7fb1d81990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 -- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 msgr2=0x7fb1d819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 0x7fb1d819d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 -- 172.21.15.19:0/3388773182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d819e100 con 0x7fb1d806b0f0 2024-04-25T15:03:31.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1d5400700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9c0 tx=0x7fb1c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400cbd0 con 0x7fb1d806b0f0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c400ee60 con 0x7fb1d806b0f0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40055c0 con 0x7fb1d806b0f0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.394+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d819e3f0 con 0x7fb1d806b0f0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.395+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d8101940 con 0x7fb1d806b0f0 2024-04-25T15:03:31.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.396+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d8069730 con 0x7fb1d806b0f0 2024-04-25T15:03:31.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.398+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb1c4004750 con 0x7fb1d806b0f0 2024-04-25T15:03:31.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.399+0000 7fb1cea00700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 0x7fb1bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:31.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.399+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb1c40d0630 con 0x7fb1d806b0f0 2024-04-25T15:03:31.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.399+0000 7fb1d5e00700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 0x7fb1bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:31.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.400+0000 7fb1d5e00700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 0x7fb1bc0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009be0 tx=0x7fb1c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.404+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1c409cfc0 con 0x7fb1d806b0f0 2024-04-25T15:03:31.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:31 smithi083 ceph-mon[32919]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:31.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:31.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:31.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:31 smithi083 ceph-mon[32919]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:31 smithi102 ceph-mon[33005]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:31 smithi102 ceph-mon[33005]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:31.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:31 smithi019 ceph-mon[28090]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:31.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:31.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:31.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:31 smithi019 ceph-mon[28090]: from='mgr.14152 172.21.15.19:0/2454946404' entity='mgr.x' 2024-04-25T15:03:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.628+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1d8062e80 con 0x7fb1bc0a3cc0 2024-04-25T15:03:31.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.640+0000 7fb1cea00700 1 -- 172.21.15.19:0/3388773182 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258857 (secure 0 0 0) 0x7fb1d8062e80 con 0x7fb1bc0a3cc0 2024-04-25T15:03:31.643 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 msgr2=0x7fb1bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 0x7fb1bc0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009be0 tx=0x7fb1c8009260 comp rx=0 tx=0).stop 2024-04-25T15:03:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 msgr2=0x7fb1d8199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9c0 tx=0x7fb1c400ecd0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 shutdown_connections 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb1bc0a3cc0 0x7fb1bc0a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1d8108a30 0x7fb1d819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1d806a750 0x7fb1d81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.645+0000 7fb1dd000700 1 --2- 172.21.15.19:0/3388773182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d806b0f0 0x7fb1d8199630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.646+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 >> 172.21.15.19:0/3388773182 conn(0x7fb1d80788f0 msgr2=0x7fb1d80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.646+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 shutdown_connections 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:31.646+0000 7fb1dd000700 1 -- 172.21.15.19:0/3388773182 wait complete. 2024-04-25T15:03:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:03:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:32 smithi019 ceph-mon[28090]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:32.455 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-25T15:03:30.038264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652181,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":317,"num_read_kb":272,"num_write":268,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":650,"ondisk_log_size":650,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383364,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558460,"statfs":{"total":1151940427776,"available":1151547863040,"internally_reserved":0,"allocated":13725696,"data_stored":7594158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004901"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015633+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:03:26.015633+0000","last_peered":"2024-04-25T15:03:26.015633+0000","last_clean":"2024-04-25T15:03:26.015633+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:03:26.015633+0000","last_undegraded":"2024-04-25T15:03:26.015633+0000","last_fullsized":"2024-04-25T15:03:26.015633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.014930+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:03:26.014930+0000","last_peered":"2024-04-25T15:03:26.014930+0000","last_clean":"2024-04-25T15:03:26.014930+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:03:26.014930+0000","last_undegraded":"2024-04-25T15:03:26.014930+0000","last_fullsized":"2024-04-25T15:03:26.014930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026768+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:03:26.026768+0000","last_peered":"2024-04-25T15:03:26.026768+0000","last_clean":"2024-04-25T15:03:26.026768+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:03:26.026768+0000","last_undegraded":"2024-04-25T15:03:26.026768+0000","last_fullsized":"2024-04-25T15:03:26.026768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021160+0000","last_change":"2024-04-25T15:02:57.474358+0000","last_active":"2024-04-25T15:03:26.021160+0000","last_peered":"2024-04-25T15:03:26.021160+0000","last_clean":"2024-04-25T15:03:26.021160+0000","last_became_active":"2024-04-25T15:02:57.473924+0000","last_became_peered":"2024-04-25T15:02:57.473924+0000","last_unstale":"2024-04-25T15:03:26.021160+0000","last_undegraded":"2024-04-25T15:03:26.021160+0000","last_fullsized":"2024-04-25T15:03:26.021160+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.681108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031274+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:03:26.031274+0000","last_peered":"2024-04-25T15:03:26.031274+0000","last_clean":"2024-04-25T15:03:26.031274+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:03:26.031274+0000","last_undegraded":"2024-04-25T15:03:26.031274+0000","last_fullsized":"2024-04-25T15:03:26.031274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022388+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:03:26.022388+0000","last_peered":"2024-04-25T15:03:26.022388+0000","last_clean":"2024-04-25T15:03:26.022388+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:03:26.022388+0000","last_undegraded":"2024-04-25T15:03:26.022388+0000","last_fullsized":"2024-04-25T15:03:26.022388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022864+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:03:26.022864+0000","last_peered":"2024-04-25T15:03:26.022864+0000","last_clean":"2024-04-25T15:03:26.022864+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:03:26.022864+0000","last_undegraded":"2024-04-25T15:03:26.022864+0000","last_fullsized":"2024-04-25T15:03:26.022864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031593+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:03:26.031593+0000","last_peered":"2024-04-25T15:03:26.031593+0000","last_clean":"2024-04-25T15:03:26.031593+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:03:26.031593+0000","last_undegraded":"2024-04-25T15:03:26.031593+0000","last_fullsized":"2024-04-25T15:03:26.031593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464309+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:02:57.464309+0000","last_peered":"2024-04-25T15:02:57.464309+0000","last_clean":"2024-04-25T15:02:57.464309+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:02:57.464309+0000","last_undegraded":"2024-04-25T15:02:57.464309+0000","last_fullsized":"2024-04-25T15:02:57.464309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003374+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:03:26.003374+0000","last_peered":"2024-04-25T15:03:26.003374+0000","last_clean":"2024-04-25T15:03:26.003374+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:03:26.003374+0000","last_undegraded":"2024-04-25T15:03:26.003374+0000","last_fullsized":"2024-04-25T15:03:26.003374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022547+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:03:26.022547+0000","last_peered":"2024-04-25T15:03:26.022547+0000","last_clean":"2024-04-25T15:03:26.022547+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:03:26.022547+0000","last_undegraded":"2024-04-25T15:03:26.022547+0000","last_fullsized":"2024-04-25T15:03:26.022547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.998394+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:03:25.998394+0000","last_peered":"2024-04-25T15:03:25.998394+0000","last_clean":"2024-04-25T15:03:25.998394+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:03:25.998394+0000","last_undegraded":"2024-04-25T15:03:25.998394+0000","last_fullsized":"2024-04-25T15:03:25.998394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017300+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:03:26.017300+0000","last_peered":"2024-04-25T15:03:26.017300+0000","last_clean":"2024-04-25T15:03:26.017300+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:03:26.017300+0000","last_undegraded":"2024-04-25T15:03:26.017300+0000","last_fullsized":"2024-04-25T15:03:26.017300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.029481+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:03:26.029481+0000","last_peered":"2024-04-25T15:03:26.029481+0000","last_clean":"2024-04-25T15:03:26.029481+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:03:26.029481+0000","last_undegraded":"2024-04-25T15:03:26.029481+0000","last_fullsized":"2024-04-25T15:03:26.029481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027337+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:03:26.027337+0000","last_peered":"2024-04-25T15:03:26.027337+0000","last_clean":"2024-04-25T15:03:26.027337+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:03:26.027337+0000","last_undegraded":"2024-04-25T15:03:26.027337+0000","last_fullsized":"2024-04-25T15:03:26.027337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463810+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:02:57.463810+0000","last_peered":"2024-04-25T15:02:57.463810+0000","last_clean":"2024-04-25T15:02:57.463810+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:02:57.463810+0000","last_undegraded":"2024-04-25T15:02:57.463810+0000","last_fullsized":"2024-04-25T15:02:57.463810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015733+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:03:26.015733+0000","last_peered":"2024-04-25T15:03:26.015733+0000","last_clean":"2024-04-25T15:03:26.015733+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:03:26.015733+0000","last_undegraded":"2024-04-25T15:03:26.015733+0000","last_fullsized":"2024-04-25T15:03:26.015733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020330+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:03:26.020330+0000","last_peered":"2024-04-25T15:03:26.020330+0000","last_clean":"2024-04-25T15:03:26.020330+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:03:26.020330+0000","last_undegraded":"2024-04-25T15:03:26.020330+0000","last_fullsized":"2024-04-25T15:03:26.020330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026069+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:03:26.026069+0000","last_peered":"2024-04-25T15:03:26.026069+0000","last_clean":"2024-04-25T15:03:26.026069+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:03:26.026069+0000","last_undegraded":"2024-04-25T15:03:26.026069+0000","last_fullsized":"2024-04-25T15:03:26.026069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.462747+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:02:57.462747+0000","last_peered":"2024-04-25T15:02:57.462747+0000","last_clean":"2024-04-25T15:02:57.462747+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:02:57.462747+0000","last_undegraded":"2024-04-25T15:02:57.462747+0000","last_fullsized":"2024-04-25T15:02:57.462747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026508+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:03:26.026508+0000","last_peered":"2024-04-25T15:03:26.026508+0000","last_clean":"2024-04-25T15:03:26.026508+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:03:26.026508+0000","last_undegraded":"2024-04-25T15:03:26.026508+0000","last_fullsized":"2024-04-25T15:03:26.026508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.526583+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:02:57.526583+0000","last_peered":"2024-04-25T15:02:57.526583+0000","last_clean":"2024-04-25T15:02:57.526583+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:02:57.526583+0000","last_undegraded":"2024-04-25T15:02:57.526583+0000","last_fullsized":"2024-04-25T15:02:57.526583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.034045+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:03:26.034045+0000","last_peered":"2024-04-25T15:03:26.034045+0000","last_clean":"2024-04-25T15:03:26.034045+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:03:26.034045+0000","last_undegraded":"2024-04-25T15:03:26.034045+0000","last_fullsized":"2024-04-25T15:03:26.034045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.476182+0000","last_change":"2024-04-25T15:02:57.476182+0000","last_active":"2024-04-25T15:02:57.476182+0000","last_peered":"2024-04-25T15:02:57.476182+0000","last_clean":"2024-04-25T15:02:57.476182+0000","last_became_active":"2024-04-25T15:02:57.472985+0000","last_became_peered":"2024-04-25T15:02:57.472985+0000","last_unstale":"2024-04-25T15:02:57.476182+0000","last_undegraded":"2024-04-25T15:02:57.476182+0000","last_fullsized":"2024-04-25T15:02:57.476182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.669296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464069+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:02:57.464069+0000","last_peered":"2024-04-25T15:02:57.464069+0000","last_clean":"2024-04-25T15:02:57.464069+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:02:57.464069+0000","last_undegraded":"2024-04-25T15:02:57.464069+0000","last_fullsized":"2024-04-25T15:02:57.464069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.468469+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:02:57.468469+0000","last_peered":"2024-04-25T15:02:57.468469+0000","last_clean":"2024-04-25T15:02:57.468469+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:02:57.468469+0000","last_undegraded":"2024-04-25T15:02:57.468469+0000","last_fullsized":"2024-04-25T15:02:57.468469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021343+0000","last_change":"2024-04-25T15:02:57.471043+0000","last_active":"2024-04-25T15:03:26.021343+0000","last_peered":"2024-04-25T15:03:26.021343+0000","last_clean":"2024-04-25T15:03:26.021343+0000","last_became_active":"2024-04-25T15:02:57.470670+0000","last_became_peered":"2024-04-25T15:02:57.470670+0000","last_unstale":"2024-04-25T15:03:26.021343+0000","last_undegraded":"2024-04-25T15:03:26.021343+0000","last_fullsized":"2024-04-25T15:03:26.021343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.243201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463718+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:02:57.463718+0000","last_peered":"2024-04-25T15:02:57.463718+0000","last_clean":"2024-04-25T15:02:57.463718+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:02:57.463718+0000","last_undegraded":"2024-04-25T15:02:57.463718+0000","last_fullsized":"2024-04-25T15:02:57.463718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022464+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:03:26.022464+0000","last_peered":"2024-04-25T15:03:26.022464+0000","last_clean":"2024-04-25T15:03:26.022464+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:03:26.022464+0000","last_undegraded":"2024-04-25T15:03:26.022464+0000","last_fullsized":"2024-04-25T15:03:26.022464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.524906+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:02:57.524906+0000","last_peered":"2024-04-25T15:02:57.524906+0000","last_clean":"2024-04-25T15:02:57.524906+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:02:57.524906+0000","last_undegraded":"2024-04-25T15:02:57.524906+0000","last_fullsized":"2024-04-25T15:02:57.524906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.032786+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:03:26.032786+0000","last_peered":"2024-04-25T15:03:26.032786+0000","last_clean":"2024-04-25T15:03:26.032786+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:03:26.032786+0000","last_undegraded":"2024-04-25T15:03:26.032786+0000","last_fullsized":"2024-04-25T15:03:26.032786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.468106+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:02:57.468106+0000","last_peered":"2024-04-25T15:02:57.468106+0000","last_clean":"2024-04-25T15:02:57.468106+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:02:57.468106+0000","last_undegraded":"2024-04-25T15:02:57.468106+0000","last_fullsized":"2024-04-25T15:02:57.468106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031130+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:03:26.031130+0000","last_peered":"2024-04-25T15:03:26.031130+0000","last_clean":"2024-04-25T15:03:26.031130+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:03:26.031130+0000","last_undegraded":"2024-04-25T15:03:26.031130+0000","last_fullsized":"2024-04-25T15:03:26.031130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463498+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:02:57.463498+0000","last_peered":"2024-04-25T15:02:57.463498+0000","last_clean":"2024-04-25T15:02:57.463498+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:02:57.463498+0000","last_undegraded":"2024-04-25T15:02:57.463498+0000","last_fullsized":"2024-04-25T15:02:57.463498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.028030+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:03:26.028030+0000","last_peered":"2024-04-25T15:03:26.028030+0000","last_clean":"2024-04-25T15:03:26.028030+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:03:26.028030+0000","last_undegraded":"2024-04-25T15:03:26.028030+0000","last_fullsized":"2024-04-25T15:03:26.028030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.525210+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:02:57.525210+0000","last_peered":"2024-04-25T15:02:57.525210+0000","last_clean":"2024-04-25T15:02:57.525210+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:02:57.525210+0000","last_undegraded":"2024-04-25T15:02:57.525210+0000","last_fullsized":"2024-04-25T15:02:57.525210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464460+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:02:57.464460+0000","last_peered":"2024-04-25T15:02:57.464460+0000","last_clean":"2024-04-25T15:02:57.464460+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:02:57.464460+0000","last_undegraded":"2024-04-25T15:02:57.464460+0000","last_fullsized":"2024-04-25T15:02:57.464460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471271+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:02:57.471271+0000","last_peered":"2024-04-25T15:02:57.471271+0000","last_clean":"2024-04-25T15:02:57.471271+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:02:57.471271+0000","last_undegraded":"2024-04-25T15:02:57.471271+0000","last_fullsized":"2024-04-25T15:02:57.471271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.033959+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:03:26.033959+0000","last_peered":"2024-04-25T15:03:26.033959+0000","last_clean":"2024-04-25T15:03:26.033959+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:03:26.033959+0000","last_undegraded":"2024-04-25T15:03:26.033959+0000","last_fullsized":"2024-04-25T15:03:26.033959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464098+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:02:57.464098+0000","last_peered":"2024-04-25T15:02:57.464098+0000","last_clean":"2024-04-25T15:02:57.464098+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:02:57.464098+0000","last_undegraded":"2024-04-25T15:02:57.464098+0000","last_fullsized":"2024-04-25T15:02:57.464098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464261+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:02:57.464261+0000","last_peered":"2024-04-25T15:02:57.464261+0000","last_clean":"2024-04-25T15:02:57.464261+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:02:57.464261+0000","last_undegraded":"2024-04-25T15:02:57.464261+0000","last_fullsized":"2024-04-25T15:02:57.464261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.486896+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:03:26.486896+0000","last_peered":"2024-04-25T15:03:26.486896+0000","last_clean":"2024-04-25T15:03:26.486896+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:03:26.486896+0000","last_undegraded":"2024-04-25T15:03:26.486896+0000","last_fullsized":"2024-04-25T15:03:26.486896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60781,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":176,"num_read_kb":148,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003707+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:03:26.003707+0000","last_peered":"2024-04-25T15:03:26.003707+0000","last_clean":"2024-04-25T15:03:26.003707+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:03:26.003707+0000","last_undegraded":"2024-04-25T15:03:26.003707+0000","last_fullsized":"2024-04-25T15:03:26.003707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471984+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:02:57.471984+0000","last_peered":"2024-04-25T15:02:57.471984+0000","last_clean":"2024-04-25T15:02:57.471984+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:02:57.471984+0000","last_undegraded":"2024-04-25T15:02:57.471984+0000","last_fullsized":"2024-04-25T15:02:57.471984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027457+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:03:26.027457+0000","last_peered":"2024-04-25T15:03:26.027457+0000","last_clean":"2024-04-25T15:03:26.027457+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:03:26.027457+0000","last_undegraded":"2024-04-25T15:03:26.027457+0000","last_fullsized":"2024-04-25T15:03:26.027457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463763+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:02:57.463763+0000","last_peered":"2024-04-25T15:02:57.463763+0000","last_clean":"2024-04-25T15:02:57.463763+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:02:57.463763+0000","last_undegraded":"2024-04-25T15:02:57.463763+0000","last_fullsized":"2024-04-25T15:02:57.463763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022245+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:03:26.022245+0000","last_peered":"2024-04-25T15:03:26.022245+0000","last_clean":"2024-04-25T15:03:26.022245+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:03:26.022245+0000","last_undegraded":"2024-04-25T15:03:26.022245+0000","last_fullsized":"2024-04-25T15:03:26.022245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467414+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:02:57.467414+0000","last_peered":"2024-04-25T15:02:57.467414+0000","last_clean":"2024-04-25T15:02:57.467414+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:02:57.467414+0000","last_undegraded":"2024-04-25T15:02:57.467414+0000","last_fullsized":"2024-04-25T15:02:57.467414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.013014+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:03:26.013014+0000","last_peered":"2024-04-25T15:03:26.013014+0000","last_clean":"2024-04-25T15:03:26.013014+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:03:26.013014+0000","last_undegraded":"2024-04-25T15:03:26.013014+0000","last_fullsized":"2024-04-25T15:03:26.013014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467617+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:02:57.467617+0000","last_peered":"2024-04-25T15:02:57.467617+0000","last_clean":"2024-04-25T15:02:57.467617+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:02:57.467617+0000","last_undegraded":"2024-04-25T15:02:57.467617+0000","last_fullsized":"2024-04-25T15:02:57.467617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.016903+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:03:26.016903+0000","last_peered":"2024-04-25T15:03:26.016903+0000","last_clean":"2024-04-25T15:03:26.016903+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:03:26.016903+0000","last_undegraded":"2024-04-25T15:03:26.016903+0000","last_fullsized":"2024-04-25T15:03:26.016903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.474354+0000","last_change":"2024-04-25T15:02:57.473783+0000","last_active":"2024-04-25T15:02:57.474354+0000","last_peered":"2024-04-25T15:02:57.474354+0000","last_clean":"2024-04-25T15:02:57.474354+0000","last_became_active":"2024-04-25T15:02:57.468466+0000","last_became_peered":"2024-04-25T15:02:57.468466+0000","last_unstale":"2024-04-25T15:02:57.474354+0000","last_undegraded":"2024-04-25T15:02:57.474354+0000","last_fullsized":"2024-04-25T15:02:57.474354+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:41.182777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003261+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:03:26.003261+0000","last_peered":"2024-04-25T15:03:26.003261+0000","last_clean":"2024-04-25T15:03:26.003261+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:03:26.003261+0000","last_undegraded":"2024-04-25T15:03:26.003261+0000","last_fullsized":"2024-04-25T15:03:26.003261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.462750+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:02:57.462750+0000","last_peered":"2024-04-25T15:02:57.462750+0000","last_clean":"2024-04-25T15:02:57.462750+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:02:57.462750+0000","last_undegraded":"2024-04-25T15:02:57.462750+0000","last_fullsized":"2024-04-25T15:02:57.462750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017233+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:03:26.017233+0000","last_peered":"2024-04-25T15:03:26.017233+0000","last_clean":"2024-04-25T15:03:26.017233+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:03:26.017233+0000","last_undegraded":"2024-04-25T15:03:26.017233+0000","last_fullsized":"2024-04-25T15:03:26.017233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471325+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:02:57.471325+0000","last_peered":"2024-04-25T15:02:57.471325+0000","last_clean":"2024-04-25T15:02:57.471325+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:02:57.471325+0000","last_undegraded":"2024-04-25T15:02:57.471325+0000","last_fullsized":"2024-04-25T15:02:57.471325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027856+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:03:26.027856+0000","last_peered":"2024-04-25T15:03:26.027856+0000","last_clean":"2024-04-25T15:03:26.027856+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:03:26.027856+0000","last_undegraded":"2024-04-25T15:03:26.027856+0000","last_fullsized":"2024-04-25T15:03:26.027856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.996539+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:03:25.996539+0000","last_peered":"2024-04-25T15:03:25.996539+0000","last_clean":"2024-04-25T15:03:25.996539+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:03:25.996539+0000","last_undegraded":"2024-04-25T15:03:25.996539+0000","last_fullsized":"2024-04-25T15:03:25.996539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467524+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:02:57.467524+0000","last_peered":"2024-04-25T15:02:57.467524+0000","last_clean":"2024-04-25T15:02:57.467524+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:02:57.467524+0000","last_undegraded":"2024-04-25T15:02:57.467524+0000","last_fullsized":"2024-04-25T15:02:57.467524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.018116+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:03:26.018116+0000","last_peered":"2024-04-25T15:03:26.018116+0000","last_clean":"2024-04-25T15:03:26.018116+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:03:26.018116+0000","last_undegraded":"2024-04-25T15:03:26.018116+0000","last_fullsized":"2024-04-25T15:03:26.018116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467581+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:02:57.467581+0000","last_peered":"2024-04-25T15:02:57.467581+0000","last_clean":"2024-04-25T15:02:57.467581+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:02:57.467581+0000","last_undegraded":"2024-04-25T15:02:57.467581+0000","last_fullsized":"2024-04-25T15:02:57.467581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471137+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:02:57.471137+0000","last_peered":"2024-04-25T15:02:57.471137+0000","last_clean":"2024-04-25T15:02:57.471137+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:02:57.471137+0000","last_undegraded":"2024-04-25T15:02:57.471137+0000","last_fullsized":"2024-04-25T15:02:57.471137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022185+0000","last_change":"2024-04-25T15:02:57.474003+0000","last_active":"2024-04-25T15:03:26.022185+0000","last_peered":"2024-04-25T15:03:26.022185+0000","last_clean":"2024-04-25T15:03:26.022185+0000","last_became_active":"2024-04-25T15:02:57.473443+0000","last_became_peered":"2024-04-25T15:02:57.473443+0000","last_unstale":"2024-04-25T15:03:26.022185+0000","last_undegraded":"2024-04-25T15:03:26.022185+0000","last_fullsized":"2024-04-25T15:03:26.022185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:30.170675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471259+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:02:57.471259+0000","last_peered":"2024-04-25T15:02:57.471259+0000","last_clean":"2024-04-25T15:02:57.471259+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:02:57.471259+0000","last_undegraded":"2024-04-25T15:02:57.471259+0000","last_fullsized":"2024-04-25T15:02:57.471259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.530180+0000","last_change":"2024-04-25T15:02:57.529669+0000","last_active":"2024-04-25T15:02:57.530180+0000","last_peered":"2024-04-25T15:02:57.530180+0000","last_clean":"2024-04-25T15:02:57.530180+0000","last_became_active":"2024-04-25T15:02:57.529390+0000","last_became_peered":"2024-04-25T15:02:57.529390+0000","last_unstale":"2024-04-25T15:02:57.530180+0000","last_undegraded":"2024-04-25T15:02:57.530180+0000","last_fullsized":"2024-04-25T15:02:57.530180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:36.255304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020868+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:03:26.020868+0000","last_peered":"2024-04-25T15:03:26.020868+0000","last_clean":"2024-04-25T15:03:26.020868+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:03:26.020868+0000","last_undegraded":"2024-04-25T15:03:26.020868+0000","last_fullsized":"2024-04-25T15:03:26.020868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463046+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:02:57.463046+0000","last_peered":"2024-04-25T15:02:57.463046+0000","last_clean":"2024-04-25T15:02:57.463046+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:02:57.463046+0000","last_undegraded":"2024-04-25T15:02:57.463046+0000","last_fullsized":"2024-04-25T15:02:57.463046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025908+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:03:26.025908+0000","last_peered":"2024-04-25T15:03:26.025908+0000","last_clean":"2024-04-25T15:03:26.025908+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:03:26.025908+0000","last_undegraded":"2024-04-25T15:03:26.025908+0000","last_fullsized":"2024-04-25T15:03:26.025908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467818+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:02:57.467818+0000","last_peered":"2024-04-25T15:02:57.467818+0000","last_clean":"2024-04-25T15:02:57.467818+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:02:57.467818+0000","last_undegraded":"2024-04-25T15:02:57.467818+0000","last_fullsized":"2024-04-25T15:02:57.467818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.032356+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:03:26.032356+0000","last_peered":"2024-04-25T15:03:26.032356+0000","last_clean":"2024-04-25T15:03:26.032356+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:03:26.032356+0000","last_undegraded":"2024-04-25T15:03:26.032356+0000","last_fullsized":"2024-04-25T15:03:26.032356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463821+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:02:57.463821+0000","last_peered":"2024-04-25T15:02:57.463821+0000","last_clean":"2024-04-25T15:02:57.463821+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:02:57.463821+0000","last_undegraded":"2024-04-25T15:02:57.463821+0000","last_fullsized":"2024-04-25T15:02:57.463821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.987597+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:03:25.987597+0000","last_peered":"2024-04-25T15:03:25.987597+0000","last_clean":"2024-04-25T15:03:25.987597+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:03:25.987597+0000","last_undegraded":"2024-04-25T15:03:25.987597+0000","last_fullsized":"2024-04-25T15:03:25.987597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.526030+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:02:57.526030+0000","last_peered":"2024-04-25T15:02:57.526030+0000","last_clean":"2024-04-25T15:02:57.526030+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:02:57.526030+0000","last_undegraded":"2024-04-25T15:02:57.526030+0000","last_fullsized":"2024-04-25T15:02:57.526030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015502+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:03:26.015502+0000","last_peered":"2024-04-25T15:03:26.015502+0000","last_clean":"2024-04-25T15:03:26.015502+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:03:26.015502+0000","last_undegraded":"2024-04-25T15:03:26.015502+0000","last_fullsized":"2024-04-25T15:03:26.015502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471668+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:02:57.471668+0000","last_peered":"2024-04-25T15:02:57.471668+0000","last_clean":"2024-04-25T15:02:57.471668+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:02:57.471668+0000","last_undegraded":"2024-04-25T15:02:57.471668+0000","last_fullsized":"2024-04-25T15:02:57.471668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.028892+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:03:26.028892+0000","last_peered":"2024-04-25T15:03:26.028892+0000","last_clean":"2024-04-25T15:03:26.028892+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:03:26.028892+0000","last_undegraded":"2024-04-25T15:03:26.028892+0000","last_fullsized":"2024-04-25T15:03:26.028892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467925+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:02:57.467925+0000","last_peered":"2024-04-25T15:02:57.467925+0000","last_clean":"2024-04-25T15:02:57.467925+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:02:57.467925+0000","last_undegraded":"2024-04-25T15:02:57.467925+0000","last_fullsized":"2024-04-25T15:02:57.467925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020381+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:03:26.020381+0000","last_peered":"2024-04-25T15:03:26.020381+0000","last_clean":"2024-04-25T15:03:26.020381+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:03:26.020381+0000","last_undegraded":"2024-04-25T15:03:26.020381+0000","last_fullsized":"2024-04-25T15:03:26.020381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.530244+0000","last_change":"2024-04-25T15:02:57.529816+0000","last_active":"2024-04-25T15:02:57.530244+0000","last_peered":"2024-04-25T15:02:57.530244+0000","last_clean":"2024-04-25T15:02:57.530244+0000","last_became_active":"2024-04-25T15:02:57.529442+0000","last_became_peered":"2024-04-25T15:02:57.529442+0000","last_unstale":"2024-04-25T15:02:57.530244+0000","last_undegraded":"2024-04-25T15:02:57.530244+0000","last_fullsized":"2024-04-25T15:02:57.530244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.143614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022709+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:03:26.022709+0000","last_peered":"2024-04-25T15:03:26.022709+0000","last_clean":"2024-04-25T15:03:26.022709+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:03:26.022709+0000","last_undegraded":"2024-04-25T15:03:26.022709+0000","last_fullsized":"2024-04-25T15:03:26.022709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463903+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:02:57.463903+0000","last_peered":"2024-04-25T15:02:57.463903+0000","last_clean":"2024-04-25T15:02:57.463903+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:02:57.463903+0000","last_undegraded":"2024-04-25T15:02:57.463903+0000","last_fullsized":"2024-04-25T15:02:57.463903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017955+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:03:26.017955+0000","last_peered":"2024-04-25T15:03:26.017955+0000","last_clean":"2024-04-25T15:03:26.017955+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:03:26.017955+0000","last_undegraded":"2024-04-25T15:03:26.017955+0000","last_fullsized":"2024-04-25T15:03:26.017955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.469923+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:02:57.469923+0000","last_peered":"2024-04-25T15:02:57.469923+0000","last_clean":"2024-04-25T15:02:57.469923+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:02:57.469923+0000","last_undegraded":"2024-04-25T15:02:57.469923+0000","last_fullsized":"2024-04-25T15:02:57.469923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006537+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:03:26.006537+0000","last_peered":"2024-04-25T15:03:26.006537+0000","last_clean":"2024-04-25T15:03:26.006537+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:03:26.006537+0000","last_undegraded":"2024-04-25T15:03:26.006537+0000","last_fullsized":"2024-04-25T15:03:26.006537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006341+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:03:26.006341+0000","last_peered":"2024-04-25T15:03:26.006341+0000","last_clean":"2024-04-25T15:03:26.006341+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:03:26.006341+0000","last_undegraded":"2024-04-25T15:03:26.006341+0000","last_fullsized":"2024-04-25T15:03:26.006341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.009130+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:03:26.009130+0000","last_peered":"2024-04-25T15:03:26.009130+0000","last_clean":"2024-04-25T15:03:26.009130+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:03:26.009130+0000","last_undegraded":"2024-04-25T15:03:26.009130+0000","last_fullsized":"2024-04-25T15:03:26.009130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025660+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:03:26.025660+0000","last_peered":"2024-04-25T15:03:26.025660+0000","last_clean":"2024-04-25T15:03:26.025660+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:03:26.025660+0000","last_undegraded":"2024-04-25T15:03:26.025660+0000","last_fullsized":"2024-04-25T15:03:26.025660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.029972+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:03:26.029972+0000","last_peered":"2024-04-25T15:03:26.029972+0000","last_clean":"2024-04-25T15:03:26.029972+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:03:26.029972+0000","last_undegraded":"2024-04-25T15:03:26.029972+0000","last_fullsized":"2024-04-25T15:03:26.029972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.019435+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:03:26.019435+0000","last_peered":"2024-04-25T15:03:26.019435+0000","last_clean":"2024-04-25T15:03:26.019435+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:03:26.019435+0000","last_undegraded":"2024-04-25T15:03:26.019435+0000","last_fullsized":"2024-04-25T15:03:26.019435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022134+0000","last_change":"2024-04-25T15:02:57.470460+0000","last_active":"2024-04-25T15:03:26.022134+0000","last_peered":"2024-04-25T15:03:26.022134+0000","last_clean":"2024-04-25T15:03:26.022134+0000","last_became_active":"2024-04-25T15:02:57.470013+0000","last_became_peered":"2024-04-25T15:02:57.470013+0000","last_unstale":"2024-04-25T15:03:26.022134+0000","last_undegraded":"2024-04-25T15:03:26.022134+0000","last_fullsized":"2024-04-25T15:03:26.022134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:07.351115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021632+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:03:26.021632+0000","last_peered":"2024-04-25T15:03:26.021632+0000","last_clean":"2024-04-25T15:03:26.021632+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:03:26.021632+0000","last_undegraded":"2024-04-25T15:03:26.021632+0000","last_fullsized":"2024-04-25T15:03:26.021632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025219+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:03:26.025219+0000","last_peered":"2024-04-25T15:03:26.025219+0000","last_clean":"2024-04-25T15:03:26.025219+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:03:26.025219+0000","last_undegraded":"2024-04-25T15:03:26.025219+0000","last_fullsized":"2024-04-25T15:03:26.025219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.033552+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:03:26.033552+0000","last_peered":"2024-04-25T15:03:26.033552+0000","last_clean":"2024-04-25T15:03:26.033552+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:03:26.033552+0000","last_undegraded":"2024-04-25T15:03:26.033552+0000","last_fullsized":"2024-04-25T15:03:26.033552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017728+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:03:26.017728+0000","last_peered":"2024-04-25T15:03:26.017728+0000","last_clean":"2024-04-25T15:03:26.017728+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:03:26.017728+0000","last_undegraded":"2024-04-25T15:03:26.017728+0000","last_fullsized":"2024-04-25T15:03:26.017728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463984+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:02:57.463984+0000","last_peered":"2024-04-25T15:02:57.463984+0000","last_clean":"2024-04-25T15:02:57.463984+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:02:57.463984+0000","last_undegraded":"2024-04-25T15:02:57.463984+0000","last_fullsized":"2024-04-25T15:02:57.463984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.024930+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:03:26.024930+0000","last_peered":"2024-04-25T15:03:26.024930+0000","last_clean":"2024-04-25T15:03:26.024930+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:03:26.024930+0000","last_undegraded":"2024-04-25T15:03:26.024930+0000","last_fullsized":"2024-04-25T15:03:26.024930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006211+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:03:26.006211+0000","last_peered":"2024-04-25T15:03:26.006211+0000","last_clean":"2024-04-25T15:03:26.006211+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:03:26.006211+0000","last_undegraded":"2024-04-25T15:03:26.006211+0000","last_fullsized":"2024-04-25T15:03:26.006211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":442,"ondisk_log_size":442,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61579,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":181,"num_read_kb":153,"num_write":85,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":184488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285599,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":469844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1121282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972403,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":470404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1572864,"data_stored":1060889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":530354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":530637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":470160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1060489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:03:32.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:03:32.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:32 smithi083 ceph-mon[32919]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:32 smithi102 ceph-mon[33005]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:32.969 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:33.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:33 smithi019 ceph-mon[28090]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:33.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:33 smithi083 ceph-mon[32919]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:33.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:33 smithi102 ceph-mon[33005]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 -- 172.21.15.19:0/1354563380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc108a30 msgr2=0x7f98cc108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 --2- 172.21.15.19:0/1354563380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc108a30 0x7f98cc108e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009f90 tx=0x7f98bc00b330 comp rx=0 tx=0).stop 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 -- 172.21.15.19:0/1354563380 shutdown_connections 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 --2- 172.21.15.19:0/1354563380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc06b0f0 0x7f98cc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 --2- 172.21.15.19:0/1354563380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc108a30 0x7f98cc108e10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 --2- 172.21.15.19:0/1354563380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 -- 172.21.15.19:0/1354563380 >> 172.21.15.19:0/1354563380 conn(0x7f98cc0788f0 msgr2=0x7f98cc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 -- 172.21.15.19:0/1354563380 shutdown_connections 2024-04-25T15:03:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.317+0000 7f98d1600700 1 -- 172.21.15.19:0/1354563380 wait complete. 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.318+0000 7f98d1600700 1 Processor -- start 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.318+0000 7f98d1600700 1 -- start start 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 0x7f98cc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112c30 con 0x7f98cc108a30 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112ab0 con 0x7f98cc06a750 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98d1600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112db0 con 0x7f98cc06b0f0 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98cb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 0x7f98cc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98c2000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98cb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 0x7f98cc19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57568/0 (socket says 172.21.15.19:57568) 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44902/0 (socket says 172.21.15.19:44902) 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98cb400700 1 -- 172.21.15.19:0/3960847605 learned_addr learned my addr 172.21.15.19:0/3960847605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98c2000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:50824/0 (socket says 172.21.15.19:50824) 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 -- 172.21.15.19:0/3960847605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 msgr2=0x7f98cc19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 -- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 msgr2=0x7f98cc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 0x7f98cc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98caa00700 1 -- 172.21.15.19:0/3960847605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc1a03f0 con 0x7f98cc06a750 2024-04-25T15:03:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.319+0000 7f98c2000700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98caa00700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000f80 tx=0x7f98bc00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc021070 con 0x7f98cc06a750 2024-04-25T15:03:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc007ac0 con 0x7f98cc06a750 2024-04-25T15:03:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc1a0680 con 0x7f98cc06a750 2024-04-25T15:03:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc005040 con 0x7f98cc06a750 2024-04-25T15:03:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.320+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a7f60 con 0x7f98cc06a750 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.321+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc069730 con 0x7f98cc06a750 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.324+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f98bc024070 con 0x7f98cc06a750 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.325+0000 7f98c3e00700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 0x7f98b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.325+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f98bc00f020 con 0x7f98cc06a750 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.325+0000 7f98c2000700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 0x7f98b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.326+0000 7f98c2000700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 0x7f98b00a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98cc078ff0 tx=0x7f98b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.328+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98bc02b020 con 0x7f98cc06a750 2024-04-25T15:03:35.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:35 smithi083 ceph-mon[32919]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.555+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98cc19c740 con 0x7f98b00a3c40 2024-04-25T15:03:35.557 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:35 smithi019 ceph-mon[28090]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:35.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:35 smithi102 ceph-mon[33005]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.578+0000 7f98c3e00700 1 -- 172.21.15.19:0/3960847605 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258827 (secure 0 0 0) 0x7f98cc19c740 con 0x7f98b00a3c40 2024-04-25T15:03:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:35.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.583+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 msgr2=0x7f98b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:35.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.583+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 0x7f98b00a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98cc078ff0 tx=0x7f98b4009260 comp rx=0 tx=0).stop 2024-04-25T15:03:35.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.583+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 msgr2=0x7f98cc19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.583+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000f80 tx=0x7f98bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 shutdown_connections 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f98b00a3c40 0x7f98b00a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98cc108a30 0x7f98cc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98cc06a750 0x7f98cc19b3e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 --2- 172.21.15.19:0/3960847605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc06b0f0 0x7f98cc19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 >> 172.21.15.19:0/3960847605 conn(0x7f98cc0788f0 msgr2=0x7f98cc10df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 shutdown_connections 2024-04-25T15:03:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:35.584+0000 7f98d1600700 1 -- 172.21.15.19:0/3960847605 wait complete. 2024-04-25T15:03:35.589 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:03:36.438 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-25T15:03:34.040202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652181,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":327,"num_read_kb":277,"num_write":268,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387516,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554308,"statfs":{"total":1151940427776,"available":1151543611392,"internally_reserved":0,"allocated":13725696,"data_stored":7594407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005360"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015633+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:03:26.015633+0000","last_peered":"2024-04-25T15:03:26.015633+0000","last_clean":"2024-04-25T15:03:26.015633+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:03:26.015633+0000","last_undegraded":"2024-04-25T15:03:26.015633+0000","last_fullsized":"2024-04-25T15:03:26.015633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.014930+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:03:26.014930+0000","last_peered":"2024-04-25T15:03:26.014930+0000","last_clean":"2024-04-25T15:03:26.014930+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:03:26.014930+0000","last_undegraded":"2024-04-25T15:03:26.014930+0000","last_fullsized":"2024-04-25T15:03:26.014930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026768+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:03:26.026768+0000","last_peered":"2024-04-25T15:03:26.026768+0000","last_clean":"2024-04-25T15:03:26.026768+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:03:26.026768+0000","last_undegraded":"2024-04-25T15:03:26.026768+0000","last_fullsized":"2024-04-25T15:03:26.026768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021160+0000","last_change":"2024-04-25T15:02:57.474358+0000","last_active":"2024-04-25T15:03:26.021160+0000","last_peered":"2024-04-25T15:03:26.021160+0000","last_clean":"2024-04-25T15:03:26.021160+0000","last_became_active":"2024-04-25T15:02:57.473924+0000","last_became_peered":"2024-04-25T15:02:57.473924+0000","last_unstale":"2024-04-25T15:03:26.021160+0000","last_undegraded":"2024-04-25T15:03:26.021160+0000","last_fullsized":"2024-04-25T15:03:26.021160+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.681108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031274+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:03:26.031274+0000","last_peered":"2024-04-25T15:03:26.031274+0000","last_clean":"2024-04-25T15:03:26.031274+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:03:26.031274+0000","last_undegraded":"2024-04-25T15:03:26.031274+0000","last_fullsized":"2024-04-25T15:03:26.031274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022388+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:03:26.022388+0000","last_peered":"2024-04-25T15:03:26.022388+0000","last_clean":"2024-04-25T15:03:26.022388+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:03:26.022388+0000","last_undegraded":"2024-04-25T15:03:26.022388+0000","last_fullsized":"2024-04-25T15:03:26.022388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022864+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:03:26.022864+0000","last_peered":"2024-04-25T15:03:26.022864+0000","last_clean":"2024-04-25T15:03:26.022864+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:03:26.022864+0000","last_undegraded":"2024-04-25T15:03:26.022864+0000","last_fullsized":"2024-04-25T15:03:26.022864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031593+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:03:26.031593+0000","last_peered":"2024-04-25T15:03:26.031593+0000","last_clean":"2024-04-25T15:03:26.031593+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:03:26.031593+0000","last_undegraded":"2024-04-25T15:03:26.031593+0000","last_fullsized":"2024-04-25T15:03:26.031593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017688+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:03:26.017688+0000","last_peered":"2024-04-25T15:03:26.017688+0000","last_clean":"2024-04-25T15:03:26.017688+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:03:26.017688+0000","last_undegraded":"2024-04-25T15:03:26.017688+0000","last_fullsized":"2024-04-25T15:03:26.017688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003374+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:03:26.003374+0000","last_peered":"2024-04-25T15:03:26.003374+0000","last_clean":"2024-04-25T15:03:26.003374+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:03:26.003374+0000","last_undegraded":"2024-04-25T15:03:26.003374+0000","last_fullsized":"2024-04-25T15:03:26.003374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022547+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:03:26.022547+0000","last_peered":"2024-04-25T15:03:26.022547+0000","last_clean":"2024-04-25T15:03:26.022547+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:03:26.022547+0000","last_undegraded":"2024-04-25T15:03:26.022547+0000","last_fullsized":"2024-04-25T15:03:26.022547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.998394+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:03:25.998394+0000","last_peered":"2024-04-25T15:03:25.998394+0000","last_clean":"2024-04-25T15:03:25.998394+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:03:25.998394+0000","last_undegraded":"2024-04-25T15:03:25.998394+0000","last_fullsized":"2024-04-25T15:03:25.998394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017300+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:03:26.017300+0000","last_peered":"2024-04-25T15:03:26.017300+0000","last_clean":"2024-04-25T15:03:26.017300+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:03:26.017300+0000","last_undegraded":"2024-04-25T15:03:26.017300+0000","last_fullsized":"2024-04-25T15:03:26.017300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.029481+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:03:26.029481+0000","last_peered":"2024-04-25T15:03:26.029481+0000","last_clean":"2024-04-25T15:03:26.029481+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:03:26.029481+0000","last_undegraded":"2024-04-25T15:03:26.029481+0000","last_fullsized":"2024-04-25T15:03:26.029481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027337+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:03:26.027337+0000","last_peered":"2024-04-25T15:03:26.027337+0000","last_clean":"2024-04-25T15:03:26.027337+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:03:26.027337+0000","last_undegraded":"2024-04-25T15:03:26.027337+0000","last_fullsized":"2024-04-25T15:03:26.027337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027581+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:03:26.027581+0000","last_peered":"2024-04-25T15:03:26.027581+0000","last_clean":"2024-04-25T15:03:26.027581+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:03:26.027581+0000","last_undegraded":"2024-04-25T15:03:26.027581+0000","last_fullsized":"2024-04-25T15:03:26.027581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015733+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:03:26.015733+0000","last_peered":"2024-04-25T15:03:26.015733+0000","last_clean":"2024-04-25T15:03:26.015733+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:03:26.015733+0000","last_undegraded":"2024-04-25T15:03:26.015733+0000","last_fullsized":"2024-04-25T15:03:26.015733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020330+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:03:26.020330+0000","last_peered":"2024-04-25T15:03:26.020330+0000","last_clean":"2024-04-25T15:03:26.020330+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:03:26.020330+0000","last_undegraded":"2024-04-25T15:03:26.020330+0000","last_fullsized":"2024-04-25T15:03:26.020330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026069+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:03:26.026069+0000","last_peered":"2024-04-25T15:03:26.026069+0000","last_clean":"2024-04-25T15:03:26.026069+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:03:26.026069+0000","last_undegraded":"2024-04-25T15:03:26.026069+0000","last_fullsized":"2024-04-25T15:03:26.026069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.462747+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:02:57.462747+0000","last_peered":"2024-04-25T15:02:57.462747+0000","last_clean":"2024-04-25T15:02:57.462747+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:02:57.462747+0000","last_undegraded":"2024-04-25T15:02:57.462747+0000","last_fullsized":"2024-04-25T15:02:57.462747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.026508+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:03:26.026508+0000","last_peered":"2024-04-25T15:03:26.026508+0000","last_clean":"2024-04-25T15:03:26.026508+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:03:26.026508+0000","last_undegraded":"2024-04-25T15:03:26.026508+0000","last_fullsized":"2024-04-25T15:03:26.026508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.526583+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:02:57.526583+0000","last_peered":"2024-04-25T15:02:57.526583+0000","last_clean":"2024-04-25T15:02:57.526583+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:02:57.526583+0000","last_undegraded":"2024-04-25T15:02:57.526583+0000","last_fullsized":"2024-04-25T15:02:57.526583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.034045+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:03:26.034045+0000","last_peered":"2024-04-25T15:03:26.034045+0000","last_clean":"2024-04-25T15:03:26.034045+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:03:26.034045+0000","last_undegraded":"2024-04-25T15:03:26.034045+0000","last_fullsized":"2024-04-25T15:03:26.034045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.476182+0000","last_change":"2024-04-25T15:02:57.476182+0000","last_active":"2024-04-25T15:02:57.476182+0000","last_peered":"2024-04-25T15:02:57.476182+0000","last_clean":"2024-04-25T15:02:57.476182+0000","last_became_active":"2024-04-25T15:02:57.472985+0000","last_became_peered":"2024-04-25T15:02:57.472985+0000","last_unstale":"2024-04-25T15:02:57.476182+0000","last_undegraded":"2024-04-25T15:02:57.476182+0000","last_fullsized":"2024-04-25T15:02:57.476182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.669296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020082+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:03:26.020082+0000","last_peered":"2024-04-25T15:03:26.020082+0000","last_clean":"2024-04-25T15:03:26.020082+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:03:26.020082+0000","last_undegraded":"2024-04-25T15:03:26.020082+0000","last_fullsized":"2024-04-25T15:03:26.020082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.468469+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:02:57.468469+0000","last_peered":"2024-04-25T15:02:57.468469+0000","last_clean":"2024-04-25T15:02:57.468469+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:02:57.468469+0000","last_undegraded":"2024-04-25T15:02:57.468469+0000","last_fullsized":"2024-04-25T15:02:57.468469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021343+0000","last_change":"2024-04-25T15:02:57.471043+0000","last_active":"2024-04-25T15:03:26.021343+0000","last_peered":"2024-04-25T15:03:26.021343+0000","last_clean":"2024-04-25T15:03:26.021343+0000","last_became_active":"2024-04-25T15:02:57.470670+0000","last_became_peered":"2024-04-25T15:02:57.470670+0000","last_unstale":"2024-04-25T15:03:26.021343+0000","last_undegraded":"2024-04-25T15:03:26.021343+0000","last_fullsized":"2024-04-25T15:03:26.021343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.243201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463718+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:02:57.463718+0000","last_peered":"2024-04-25T15:02:57.463718+0000","last_clean":"2024-04-25T15:02:57.463718+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:02:57.463718+0000","last_undegraded":"2024-04-25T15:02:57.463718+0000","last_fullsized":"2024-04-25T15:02:57.463718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022464+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:03:26.022464+0000","last_peered":"2024-04-25T15:03:26.022464+0000","last_clean":"2024-04-25T15:03:26.022464+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:03:26.022464+0000","last_undegraded":"2024-04-25T15:03:26.022464+0000","last_fullsized":"2024-04-25T15:03:26.022464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.524906+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:02:57.524906+0000","last_peered":"2024-04-25T15:02:57.524906+0000","last_clean":"2024-04-25T15:02:57.524906+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:02:57.524906+0000","last_undegraded":"2024-04-25T15:02:57.524906+0000","last_fullsized":"2024-04-25T15:02:57.524906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.032786+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:03:26.032786+0000","last_peered":"2024-04-25T15:03:26.032786+0000","last_clean":"2024-04-25T15:03:26.032786+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:03:26.032786+0000","last_undegraded":"2024-04-25T15:03:26.032786+0000","last_fullsized":"2024-04-25T15:03:26.032786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.468106+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:02:57.468106+0000","last_peered":"2024-04-25T15:02:57.468106+0000","last_clean":"2024-04-25T15:02:57.468106+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:02:57.468106+0000","last_undegraded":"2024-04-25T15:02:57.468106+0000","last_fullsized":"2024-04-25T15:02:57.468106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.031130+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:03:26.031130+0000","last_peered":"2024-04-25T15:03:26.031130+0000","last_clean":"2024-04-25T15:03:26.031130+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:03:26.031130+0000","last_undegraded":"2024-04-25T15:03:26.031130+0000","last_fullsized":"2024-04-25T15:03:26.031130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463498+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:02:57.463498+0000","last_peered":"2024-04-25T15:02:57.463498+0000","last_clean":"2024-04-25T15:02:57.463498+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:02:57.463498+0000","last_undegraded":"2024-04-25T15:02:57.463498+0000","last_fullsized":"2024-04-25T15:02:57.463498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.028030+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:03:26.028030+0000","last_peered":"2024-04-25T15:03:26.028030+0000","last_clean":"2024-04-25T15:03:26.028030+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:03:26.028030+0000","last_undegraded":"2024-04-25T15:03:26.028030+0000","last_fullsized":"2024-04-25T15:03:26.028030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.525210+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:02:57.525210+0000","last_peered":"2024-04-25T15:02:57.525210+0000","last_clean":"2024-04-25T15:02:57.525210+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:02:57.525210+0000","last_undegraded":"2024-04-25T15:02:57.525210+0000","last_fullsized":"2024-04-25T15:02:57.525210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021324+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:03:26.021324+0000","last_peered":"2024-04-25T15:03:26.021324+0000","last_clean":"2024-04-25T15:03:26.021324+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:03:26.021324+0000","last_undegraded":"2024-04-25T15:03:26.021324+0000","last_fullsized":"2024-04-25T15:03:26.021324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471271+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:02:57.471271+0000","last_peered":"2024-04-25T15:02:57.471271+0000","last_clean":"2024-04-25T15:02:57.471271+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:02:57.471271+0000","last_undegraded":"2024-04-25T15:02:57.471271+0000","last_fullsized":"2024-04-25T15:02:57.471271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.033959+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:03:26.033959+0000","last_peered":"2024-04-25T15:03:26.033959+0000","last_clean":"2024-04-25T15:03:26.033959+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:03:26.033959+0000","last_undegraded":"2024-04-25T15:03:26.033959+0000","last_fullsized":"2024-04-25T15:03:26.033959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.464098+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:02:57.464098+0000","last_peered":"2024-04-25T15:02:57.464098+0000","last_clean":"2024-04-25T15:02:57.464098+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:02:57.464098+0000","last_undegraded":"2024-04-25T15:02:57.464098+0000","last_fullsized":"2024-04-25T15:02:57.464098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021310+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:03:26.021310+0000","last_peered":"2024-04-25T15:03:26.021310+0000","last_clean":"2024-04-25T15:03:26.021310+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:03:26.021310+0000","last_undegraded":"2024-04-25T15:03:26.021310+0000","last_fullsized":"2024-04-25T15:03:26.021310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":235,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:31.497262+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:03:31.497262+0000","last_peered":"2024-04-25T15:03:31.497262+0000","last_clean":"2024-04-25T15:03:31.497262+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:03:31.497262+0000","last_undegraded":"2024-04-25T15:03:31.497262+0000","last_fullsized":"2024-04-25T15:03:31.497262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60781,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":186,"num_read_kb":153,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003707+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:03:26.003707+0000","last_peered":"2024-04-25T15:03:26.003707+0000","last_clean":"2024-04-25T15:03:26.003707+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:03:26.003707+0000","last_undegraded":"2024-04-25T15:03:26.003707+0000","last_fullsized":"2024-04-25T15:03:26.003707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471984+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:02:57.471984+0000","last_peered":"2024-04-25T15:02:57.471984+0000","last_clean":"2024-04-25T15:02:57.471984+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:02:57.471984+0000","last_undegraded":"2024-04-25T15:02:57.471984+0000","last_fullsized":"2024-04-25T15:02:57.471984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027457+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:03:26.027457+0000","last_peered":"2024-04-25T15:03:26.027457+0000","last_clean":"2024-04-25T15:03:26.027457+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:03:26.027457+0000","last_undegraded":"2024-04-25T15:03:26.027457+0000","last_fullsized":"2024-04-25T15:03:26.027457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463763+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:02:57.463763+0000","last_peered":"2024-04-25T15:02:57.463763+0000","last_clean":"2024-04-25T15:02:57.463763+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:02:57.463763+0000","last_undegraded":"2024-04-25T15:02:57.463763+0000","last_fullsized":"2024-04-25T15:02:57.463763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022245+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:03:26.022245+0000","last_peered":"2024-04-25T15:03:26.022245+0000","last_clean":"2024-04-25T15:03:26.022245+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:03:26.022245+0000","last_undegraded":"2024-04-25T15:03:26.022245+0000","last_fullsized":"2024-04-25T15:03:26.022245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467414+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:02:57.467414+0000","last_peered":"2024-04-25T15:02:57.467414+0000","last_clean":"2024-04-25T15:02:57.467414+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:02:57.467414+0000","last_undegraded":"2024-04-25T15:02:57.467414+0000","last_fullsized":"2024-04-25T15:02:57.467414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.013014+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:03:26.013014+0000","last_peered":"2024-04-25T15:03:26.013014+0000","last_clean":"2024-04-25T15:03:26.013014+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:03:26.013014+0000","last_undegraded":"2024-04-25T15:03:26.013014+0000","last_fullsized":"2024-04-25T15:03:26.013014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467617+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:02:57.467617+0000","last_peered":"2024-04-25T15:02:57.467617+0000","last_clean":"2024-04-25T15:02:57.467617+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:02:57.467617+0000","last_undegraded":"2024-04-25T15:02:57.467617+0000","last_fullsized":"2024-04-25T15:02:57.467617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.016903+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:03:26.016903+0000","last_peered":"2024-04-25T15:03:26.016903+0000","last_clean":"2024-04-25T15:03:26.016903+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:03:26.016903+0000","last_undegraded":"2024-04-25T15:03:26.016903+0000","last_fullsized":"2024-04-25T15:03:26.016903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.474354+0000","last_change":"2024-04-25T15:02:57.473783+0000","last_active":"2024-04-25T15:02:57.474354+0000","last_peered":"2024-04-25T15:02:57.474354+0000","last_clean":"2024-04-25T15:02:57.474354+0000","last_became_active":"2024-04-25T15:02:57.468466+0000","last_became_peered":"2024-04-25T15:02:57.468466+0000","last_unstale":"2024-04-25T15:02:57.474354+0000","last_undegraded":"2024-04-25T15:02:57.474354+0000","last_fullsized":"2024-04-25T15:02:57.474354+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:41.182777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.003261+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:03:26.003261+0000","last_peered":"2024-04-25T15:03:26.003261+0000","last_clean":"2024-04-25T15:03:26.003261+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:03:26.003261+0000","last_undegraded":"2024-04-25T15:03:26.003261+0000","last_fullsized":"2024-04-25T15:03:26.003261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.462750+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:02:57.462750+0000","last_peered":"2024-04-25T15:02:57.462750+0000","last_clean":"2024-04-25T15:02:57.462750+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:02:57.462750+0000","last_undegraded":"2024-04-25T15:02:57.462750+0000","last_fullsized":"2024-04-25T15:02:57.462750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017233+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:03:26.017233+0000","last_peered":"2024-04-25T15:03:26.017233+0000","last_clean":"2024-04-25T15:03:26.017233+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:03:26.017233+0000","last_undegraded":"2024-04-25T15:03:26.017233+0000","last_fullsized":"2024-04-25T15:03:26.017233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471325+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:02:57.471325+0000","last_peered":"2024-04-25T15:02:57.471325+0000","last_clean":"2024-04-25T15:02:57.471325+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:02:57.471325+0000","last_undegraded":"2024-04-25T15:02:57.471325+0000","last_fullsized":"2024-04-25T15:02:57.471325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027856+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:03:26.027856+0000","last_peered":"2024-04-25T15:03:26.027856+0000","last_clean":"2024-04-25T15:03:26.027856+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:03:26.027856+0000","last_undegraded":"2024-04-25T15:03:26.027856+0000","last_fullsized":"2024-04-25T15:03:26.027856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.996539+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:03:25.996539+0000","last_peered":"2024-04-25T15:03:25.996539+0000","last_clean":"2024-04-25T15:03:25.996539+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:03:25.996539+0000","last_undegraded":"2024-04-25T15:03:25.996539+0000","last_fullsized":"2024-04-25T15:03:25.996539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467524+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:02:57.467524+0000","last_peered":"2024-04-25T15:02:57.467524+0000","last_clean":"2024-04-25T15:02:57.467524+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:02:57.467524+0000","last_undegraded":"2024-04-25T15:02:57.467524+0000","last_fullsized":"2024-04-25T15:02:57.467524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.018116+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:03:26.018116+0000","last_peered":"2024-04-25T15:03:26.018116+0000","last_clean":"2024-04-25T15:03:26.018116+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:03:26.018116+0000","last_undegraded":"2024-04-25T15:03:26.018116+0000","last_fullsized":"2024-04-25T15:03:26.018116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467581+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:02:57.467581+0000","last_peered":"2024-04-25T15:02:57.467581+0000","last_clean":"2024-04-25T15:02:57.467581+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:02:57.467581+0000","last_undegraded":"2024-04-25T15:02:57.467581+0000","last_fullsized":"2024-04-25T15:02:57.467581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471137+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:02:57.471137+0000","last_peered":"2024-04-25T15:02:57.471137+0000","last_clean":"2024-04-25T15:02:57.471137+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:02:57.471137+0000","last_undegraded":"2024-04-25T15:02:57.471137+0000","last_fullsized":"2024-04-25T15:02:57.471137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022185+0000","last_change":"2024-04-25T15:02:57.474003+0000","last_active":"2024-04-25T15:03:26.022185+0000","last_peered":"2024-04-25T15:03:26.022185+0000","last_clean":"2024-04-25T15:03:26.022185+0000","last_became_active":"2024-04-25T15:02:57.473443+0000","last_became_peered":"2024-04-25T15:02:57.473443+0000","last_unstale":"2024-04-25T15:03:26.022185+0000","last_undegraded":"2024-04-25T15:03:26.022185+0000","last_fullsized":"2024-04-25T15:03:26.022185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:30.170675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471259+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:02:57.471259+0000","last_peered":"2024-04-25T15:02:57.471259+0000","last_clean":"2024-04-25T15:02:57.471259+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:02:57.471259+0000","last_undegraded":"2024-04-25T15:02:57.471259+0000","last_fullsized":"2024-04-25T15:02:57.471259+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.530180+0000","last_change":"2024-04-25T15:02:57.529669+0000","last_active":"2024-04-25T15:02:57.530180+0000","last_peered":"2024-04-25T15:02:57.530180+0000","last_clean":"2024-04-25T15:02:57.530180+0000","last_became_active":"2024-04-25T15:02:57.529390+0000","last_became_peered":"2024-04-25T15:02:57.529390+0000","last_unstale":"2024-04-25T15:02:57.530180+0000","last_undegraded":"2024-04-25T15:02:57.530180+0000","last_fullsized":"2024-04-25T15:02:57.530180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:36.255304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020868+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:03:26.020868+0000","last_peered":"2024-04-25T15:03:26.020868+0000","last_clean":"2024-04-25T15:03:26.020868+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:03:26.020868+0000","last_undegraded":"2024-04-25T15:03:26.020868+0000","last_fullsized":"2024-04-25T15:03:26.020868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463046+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:02:57.463046+0000","last_peered":"2024-04-25T15:02:57.463046+0000","last_clean":"2024-04-25T15:02:57.463046+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:02:57.463046+0000","last_undegraded":"2024-04-25T15:02:57.463046+0000","last_fullsized":"2024-04-25T15:02:57.463046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025908+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:03:26.025908+0000","last_peered":"2024-04-25T15:03:26.025908+0000","last_clean":"2024-04-25T15:03:26.025908+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:03:26.025908+0000","last_undegraded":"2024-04-25T15:03:26.025908+0000","last_fullsized":"2024-04-25T15:03:26.025908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467818+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:02:57.467818+0000","last_peered":"2024-04-25T15:02:57.467818+0000","last_clean":"2024-04-25T15:02:57.467818+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:02:57.467818+0000","last_undegraded":"2024-04-25T15:02:57.467818+0000","last_fullsized":"2024-04-25T15:02:57.467818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.032356+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:03:26.032356+0000","last_peered":"2024-04-25T15:03:26.032356+0000","last_clean":"2024-04-25T15:03:26.032356+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:03:26.032356+0000","last_undegraded":"2024-04-25T15:03:26.032356+0000","last_fullsized":"2024-04-25T15:03:26.032356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463821+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:02:57.463821+0000","last_peered":"2024-04-25T15:02:57.463821+0000","last_clean":"2024-04-25T15:02:57.463821+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:02:57.463821+0000","last_undegraded":"2024-04-25T15:02:57.463821+0000","last_fullsized":"2024-04-25T15:02:57.463821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:25.987597+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:03:25.987597+0000","last_peered":"2024-04-25T15:03:25.987597+0000","last_clean":"2024-04-25T15:03:25.987597+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:03:25.987597+0000","last_undegraded":"2024-04-25T15:03:25.987597+0000","last_fullsized":"2024-04-25T15:03:25.987597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.526030+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:02:57.526030+0000","last_peered":"2024-04-25T15:02:57.526030+0000","last_clean":"2024-04-25T15:02:57.526030+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:02:57.526030+0000","last_undegraded":"2024-04-25T15:02:57.526030+0000","last_fullsized":"2024-04-25T15:02:57.526030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.015502+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:03:26.015502+0000","last_peered":"2024-04-25T15:03:26.015502+0000","last_clean":"2024-04-25T15:03:26.015502+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:03:26.015502+0000","last_undegraded":"2024-04-25T15:03:26.015502+0000","last_fullsized":"2024-04-25T15:03:26.015502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.471668+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:02:57.471668+0000","last_peered":"2024-04-25T15:02:57.471668+0000","last_clean":"2024-04-25T15:02:57.471668+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:02:57.471668+0000","last_undegraded":"2024-04-25T15:02:57.471668+0000","last_fullsized":"2024-04-25T15:02:57.471668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.028892+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:03:26.028892+0000","last_peered":"2024-04-25T15:03:26.028892+0000","last_clean":"2024-04-25T15:03:26.028892+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:03:26.028892+0000","last_undegraded":"2024-04-25T15:03:26.028892+0000","last_fullsized":"2024-04-25T15:03:26.028892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.467925+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:02:57.467925+0000","last_peered":"2024-04-25T15:02:57.467925+0000","last_clean":"2024-04-25T15:02:57.467925+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:02:57.467925+0000","last_undegraded":"2024-04-25T15:02:57.467925+0000","last_fullsized":"2024-04-25T15:02:57.467925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.020381+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:03:26.020381+0000","last_peered":"2024-04-25T15:03:26.020381+0000","last_clean":"2024-04-25T15:03:26.020381+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:03:26.020381+0000","last_undegraded":"2024-04-25T15:03:26.020381+0000","last_fullsized":"2024-04-25T15:03:26.020381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.530244+0000","last_change":"2024-04-25T15:02:57.529816+0000","last_active":"2024-04-25T15:02:57.530244+0000","last_peered":"2024-04-25T15:02:57.530244+0000","last_clean":"2024-04-25T15:02:57.530244+0000","last_became_active":"2024-04-25T15:02:57.529442+0000","last_became_peered":"2024-04-25T15:02:57.529442+0000","last_unstale":"2024-04-25T15:02:57.530244+0000","last_undegraded":"2024-04-25T15:02:57.530244+0000","last_fullsized":"2024-04-25T15:02:57.530244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.143614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022709+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:03:26.022709+0000","last_peered":"2024-04-25T15:03:26.022709+0000","last_clean":"2024-04-25T15:03:26.022709+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:03:26.022709+0000","last_undegraded":"2024-04-25T15:03:26.022709+0000","last_fullsized":"2024-04-25T15:03:26.022709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.463903+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:02:57.463903+0000","last_peered":"2024-04-25T15:02:57.463903+0000","last_clean":"2024-04-25T15:02:57.463903+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:02:57.463903+0000","last_undegraded":"2024-04-25T15:02:57.463903+0000","last_fullsized":"2024-04-25T15:02:57.463903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017955+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:03:26.017955+0000","last_peered":"2024-04-25T15:03:26.017955+0000","last_clean":"2024-04-25T15:03:26.017955+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:03:26.017955+0000","last_undegraded":"2024-04-25T15:03:26.017955+0000","last_fullsized":"2024-04-25T15:03:26.017955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:02:57.469923+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:02:57.469923+0000","last_peered":"2024-04-25T15:02:57.469923+0000","last_clean":"2024-04-25T15:02:57.469923+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:02:57.469923+0000","last_undegraded":"2024-04-25T15:02:57.469923+0000","last_fullsized":"2024-04-25T15:02:57.469923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006537+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:03:26.006537+0000","last_peered":"2024-04-25T15:03:26.006537+0000","last_clean":"2024-04-25T15:03:26.006537+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:03:26.006537+0000","last_undegraded":"2024-04-25T15:03:26.006537+0000","last_fullsized":"2024-04-25T15:03:26.006537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006341+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:03:26.006341+0000","last_peered":"2024-04-25T15:03:26.006341+0000","last_clean":"2024-04-25T15:03:26.006341+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:03:26.006341+0000","last_undegraded":"2024-04-25T15:03:26.006341+0000","last_fullsized":"2024-04-25T15:03:26.006341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.009130+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:03:26.009130+0000","last_peered":"2024-04-25T15:03:26.009130+0000","last_clean":"2024-04-25T15:03:26.009130+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:03:26.009130+0000","last_undegraded":"2024-04-25T15:03:26.009130+0000","last_fullsized":"2024-04-25T15:03:26.009130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025660+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:03:26.025660+0000","last_peered":"2024-04-25T15:03:26.025660+0000","last_clean":"2024-04-25T15:03:26.025660+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:03:26.025660+0000","last_undegraded":"2024-04-25T15:03:26.025660+0000","last_fullsized":"2024-04-25T15:03:26.025660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.029972+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:03:26.029972+0000","last_peered":"2024-04-25T15:03:26.029972+0000","last_clean":"2024-04-25T15:03:26.029972+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:03:26.029972+0000","last_undegraded":"2024-04-25T15:03:26.029972+0000","last_fullsized":"2024-04-25T15:03:26.029972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.019435+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:03:26.019435+0000","last_peered":"2024-04-25T15:03:26.019435+0000","last_clean":"2024-04-25T15:03:26.019435+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:03:26.019435+0000","last_undegraded":"2024-04-25T15:03:26.019435+0000","last_fullsized":"2024-04-25T15:03:26.019435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.022134+0000","last_change":"2024-04-25T15:02:57.470460+0000","last_active":"2024-04-25T15:03:26.022134+0000","last_peered":"2024-04-25T15:03:26.022134+0000","last_clean":"2024-04-25T15:03:26.022134+0000","last_became_active":"2024-04-25T15:02:57.470013+0000","last_became_peered":"2024-04-25T15:02:57.470013+0000","last_unstale":"2024-04-25T15:03:26.022134+0000","last_undegraded":"2024-04-25T15:03:26.022134+0000","last_fullsized":"2024-04-25T15:03:26.022134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:07.351115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.021632+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:03:26.021632+0000","last_peered":"2024-04-25T15:03:26.021632+0000","last_clean":"2024-04-25T15:03:26.021632+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:03:26.021632+0000","last_undegraded":"2024-04-25T15:03:26.021632+0000","last_fullsized":"2024-04-25T15:03:26.021632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.025219+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:03:26.025219+0000","last_peered":"2024-04-25T15:03:26.025219+0000","last_clean":"2024-04-25T15:03:26.025219+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:03:26.025219+0000","last_undegraded":"2024-04-25T15:03:26.025219+0000","last_fullsized":"2024-04-25T15:03:26.025219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.033552+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:03:26.033552+0000","last_peered":"2024-04-25T15:03:26.033552+0000","last_clean":"2024-04-25T15:03:26.033552+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:03:26.033552+0000","last_undegraded":"2024-04-25T15:03:26.033552+0000","last_fullsized":"2024-04-25T15:03:26.033552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.017728+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:03:26.017728+0000","last_peered":"2024-04-25T15:03:26.017728+0000","last_clean":"2024-04-25T15:03:26.017728+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:03:26.017728+0000","last_undegraded":"2024-04-25T15:03:26.017728+0000","last_fullsized":"2024-04-25T15:03:26.017728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.027550+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:03:26.027550+0000","last_peered":"2024-04-25T15:03:26.027550+0000","last_clean":"2024-04-25T15:03:26.027550+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:03:26.027550+0000","last_undegraded":"2024-04-25T15:03:26.027550+0000","last_fullsized":"2024-04-25T15:03:26.027550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.024930+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:03:26.024930+0000","last_peered":"2024-04-25T15:03:26.024930+0000","last_clean":"2024-04-25T15:03:26.024930+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:03:26.024930+0000","last_undegraded":"2024-04-25T15:03:26.024930+0000","last_fullsized":"2024-04-25T15:03:26.024930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T15:03:26.006211+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:03:26.006211+0000","last_peered":"2024-04-25T15:03:26.006211+0000","last_clean":"2024-04-25T15:03:26.006211+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:03:26.006211+0000","last_undegraded":"2024-04-25T15:03:26.006211+0000","last_fullsized":"2024-04-25T15:03:26.006211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61579,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":191,"num_read_kb":158,"num_write":85,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":184737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285600,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":469844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1121282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":470404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972403,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":470404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1572864,"data_stored":1060889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":530603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":530637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":470160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":469844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1060489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:03:36.442 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T15:03:36.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T15:03:36.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T15:03:36.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T15:03:36.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:36 smithi083 ceph-mon[32919]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:36.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:36 smithi102 ceph-mon[33005]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:36.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:36 smithi019 ceph-mon[28090]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:03:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:37.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:37 smithi019 ceph-mon[28090]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:37.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:37 smithi083 ceph-mon[32919]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:37.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:37 smithi102 ceph-mon[33005]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.447+0000 7f5a2de00700 1 -- 172.21.15.19:0/2967819791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 msgr2=0x7f5a2810f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.447+0000 7f5a2de00700 1 --2- 172.21.15.19:0/2967819791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2810f820 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b0a0 tx=0x7f5a2000b3b0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 -- 172.21.15.19:0/2967819791 shutdown_connections 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 --2- 172.21.15.19:0/2967819791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2810f820 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 --2- 172.21.15.19:0/2967819791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28108950 0x7f5a28108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 --2- 172.21.15.19:0/2967819791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28102950 0x7f5a28102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 -- 172.21.15.19:0/2967819791 >> 172.21.15.19:0/2967819791 conn(0x7f5a280fe460 msgr2=0x7f5a28100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 -- 172.21.15.19:0/2967819791 shutdown_connections 2024-04-25T15:03:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.448+0000 7f5a2de00700 1 -- 172.21.15.19:0/2967819791 wait complete. 2024-04-25T15:03:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.449+0000 7f5a2de00700 1 Processor -- start 2024-04-25T15:03:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.449+0000 7f5a2de00700 1 -- start start 2024-04-25T15:03:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 0x7f5a2819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28112be0 con 0x7f5a28108950 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28112a60 con 0x7f5a28102950 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2de00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28112d60 con 0x7f5a28103310 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a27e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 0x7f5a2819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44926/0 (socket says 172.21.15.19:44926) 2024-04-25T15:03:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a2ca00700 1 -- 172.21.15.19:0/1707302316 learned_addr learned my addr 172.21.15.19:0/1707302316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a27e00700 1 -- 172.21.15.19:0/1707302316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 msgr2=0x7f5a2819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.450+0000 7f5a27e00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a27e00700 1 -- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 msgr2=0x7f5a2819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a27e00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 0x7f5a2819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a27e00700 1 -- 172.21.15.19:0/1707302316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a281a01c0 con 0x7f5a28103310 2024-04-25T15:03:39.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a2ca00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a2d400700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 0x7f5a2819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a27e00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2819b6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400e9f0 tx=0x7f5a1400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400c9c0 con 0x7f5a28103310 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1400cb60 con 0x7f5a28103310 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a281a04b0 con 0x7f5a28103310 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14018680 con 0x7f5a28103310 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.451+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281a7d30 con 0x7f5a28103310 2024-04-25T15:03:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.453+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28069730 con 0x7f5a28103310 2024-04-25T15:03:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.454+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a14010430 con 0x7f5a28103310 2024-04-25T15:03:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.456+0000 7f5a25600700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 0x7f5a100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.456+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5a140d0530 con 0x7f5a28103310 2024-04-25T15:03:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.456+0000 7f5a2ca00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 0x7f5a100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.457+0000 7f5a2ca00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 0x7f5a100a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c011550 tx=0x7f5a1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.460+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a1409cec0 con 0x7f5a28103310 2024-04-25T15:03:39.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:39 smithi083 ceph-mon[32919]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:39.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:39 smithi102 ceph-mon[33005]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:39.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:39 smithi019 ceph-mon[28090]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.739+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5a281a7f60 con 0x7f5a28103310 2024-04-25T15:03:39.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.740+0000 7f5a25600700 1 -- 172.21.15.19:0/1707302316 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5a1409c7c0 con 0x7f5a28103310 2024-04-25T15:03:39.742 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:39.742 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T15:03:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.743+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 msgr2=0x7f5a100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.743+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 0x7f5a100a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c011550 tx=0x7f5a1c009260 comp rx=0 tx=0).stop 2024-04-25T15:03:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.743+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 msgr2=0x7f5a2819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.743+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2819b6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400e9f0 tx=0x7f5a1400ed00 comp rx=0 tx=0).stop 2024-04-25T15:03:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 shutdown_connections 2024-04-25T15:03:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f5a100a3cc0 0x7f5a100a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a28108950 0x7f5a2819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a28102950 0x7f5a2819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 --2- 172.21.15.19:0/1707302316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a28103310 0x7f5a2819b6f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 >> 172.21.15.19:0/1707302316 conn(0x7f5a280fe460 msgr2=0x7f5a280750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 shutdown_connections 2024-04-25T15:03:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:39.744+0000 7f5a2de00700 1 -- 172.21.15.19:0/1707302316 wait complete. 2024-04-25T15:03:40.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:40 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1707302316' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:03:40.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:40 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1707302316' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:03:40.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:40 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1707302316' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:03:40.699 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T15:03:40.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 -- 172.21.15.19:0/345115545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101083d0 msgr2=0x7f2f101087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 --2- 172.21.15.19:0/345115545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101083d0 0x7f2f101087f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009860 tx=0x7f2f0402a340 comp rx=0 tx=0).stop 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 -- 172.21.15.19:0/345115545 shutdown_connections 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 --2- 172.21.15.19:0/345115545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f10110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 --2- 172.21.15.19:0/345115545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101095c0 0x7f2f10109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 --2- 172.21.15.19:0/345115545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101083d0 0x7f2f101087f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 -- 172.21.15.19:0/345115545 >> 172.21.15.19:0/345115545 conn(0x7f2f10077390 msgr2=0x7f2f100797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.922+0000 7f2f16800640 1 -- 172.21.15.19:0/345115545 shutdown_connections 2024-04-25T15:03:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.923+0000 7f2f16800640 1 -- 172.21.15.19:0/345115545 wait complete. 2024-04-25T15:03:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.924+0000 7f2f16800640 1 Processor -- start 2024-04-25T15:03:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.924+0000 7f2f16800640 1 -- start start 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 0x7f2f1019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f1019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101a3dd0 0x7f2f101a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10113ee0 con 0x7f2f10109f80 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10113d60 con 0x7f2f101095c0 2024-04-25T15:03:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f16800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10114060 con 0x7f2f101a3dd0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f14a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f1019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f14a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f1019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57604/0 (socket says 172.21.15.19:57604) 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 0x7f2f1019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f14a00640 1 -- 172.21.15.19:0/3656606392 learned_addr learned my addr 172.21.15.19:0/3656606392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15e00640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101a3dd0 0x7f2f101a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 -- 172.21.15.19:0/3656606392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101a3dd0 msgr2=0x7f2f101a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101a3dd0 0x7f2f101a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 -- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 msgr2=0x7f2f1019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f1019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.925+0000 7f2f15400640 1 -- 172.21.15.19:0/3656606392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f101a6830 con 0x7f2f101095c0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f15400640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 0x7f2f1019e880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f10107f10 tx=0x7f2f04032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04037040 con 0x7f2f101095c0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f04030dd0 con 0x7f2f101095c0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f101a6ac0 con 0x7f2f101095c0 2024-04-25T15:03:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004990 con 0x7f2f101095c0 2024-04-25T15:03:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.926+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f101a6fd0 con 0x7f2f101095c0 2024-04-25T15:03:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.928+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1010f8b0 con 0x7f2f101095c0 2024-04-25T15:03:40.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.929+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f04005140 con 0x7f2f101095c0 2024-04-25T15:03:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.931+0000 7f2f0e000640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 0x7f2ef80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.931+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2f040ef870 con 0x7f2f101095c0 2024-04-25T15:03:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.931+0000 7f2f14a00640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 0x7f2ef80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:40.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.932+0000 7f2f14a00640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 0x7f2ef80a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f1019fd40 tx=0x7f2efc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:40.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:40.935+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f040bc200 con 0x7f2f101095c0 2024-04-25T15:03:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.145+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f10062b50 con 0x7f2f101095c0 2024-04-25T15:03:41.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.148+0000 7f2f0e000640 1 -- 172.21.15.19:0/3656606392 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7f2f040bbb00 con 0x7f2f101095c0 2024-04-25T15:03:41.149 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:41.149 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:03:41.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 msgr2=0x7f2ef80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 0x7f2ef80a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f1019fd40 tx=0x7f2efc009260 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 msgr2=0x7f2f1019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 0x7f2f1019e880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f10107f10 tx=0x7f2f04032dc0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 shutdown_connections 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f101a3dd0 0x7f2f101a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f10109f80 0x7f2f1019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2ef80a3c30 0x7f2ef80a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 --2- 172.21.15.19:0/3656606392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f101095c0 0x7f2f1019e880 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 >> 172.21.15.19:0/3656606392 conn(0x7f2f10077390 msgr2=0x7f2f1010ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.150+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 shutdown_connections 2024-04-25T15:03:41.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.151+0000 7f2f16800640 1 -- 172.21.15.19:0/3656606392 wait complete. 2024-04-25T15:03:41.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:03:41.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 -- 172.21.15.19:0/807648796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff841004f0 msgr2=0x7eff84102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 --2- 172.21.15.19:0/807648796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff841004f0 0x7eff84102100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff80009ca0 tx=0x7eff8002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 -- 172.21.15.19:0/807648796 shutdown_connections 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 --2- 172.21.15.19:0/807648796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff841004f0 0x7eff84102100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 --2- 172.21.15.19:0/807648796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff840fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 --2- 172.21.15.19:0/807648796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff84101060 0x7eff84101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 -- 172.21.15.19:0/807648796 >> 172.21.15.19:0/807648796 conn(0x7eff840fb610 msgr2=0x7eff840fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 -- 172.21.15.19:0/807648796 shutdown_connections 2024-04-25T15:03:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.396+0000 7eff8b200640 1 -- 172.21.15.19:0/807648796 wait complete. 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 Processor -- start 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 -- start start 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 0x7eff841a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410fbe0 con 0x7eff840ffb30 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410fa60 con 0x7eff84101060 2024-04-25T15:03:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.397+0000 7eff8b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410fd60 con 0x7eff841004f0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff8a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 0x7eff841a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:50868/0 (socket says 172.21.15.19:50868) 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57614/0 (socket says 172.21.15.19:57614) 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89e00640 1 -- 172.21.15.19:0/833075115 learned_addr learned my addr 172.21.15.19:0/833075115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff8a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 0x7eff841a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44948/0 (socket says 172.21.15.19:44948) 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 -- 172.21.15.19:0/833075115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 msgr2=0x7eff841a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 0x7eff841a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 -- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 msgr2=0x7eff8419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89400640 1 -- 172.21.15.19:0/833075115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841a1eb0 con 0x7eff841004f0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.398+0000 7eff89e00640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff89400640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9c0 tx=0x7eff7800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:41.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800ccc0 con 0x7eff841004f0 2024-04-25T15:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff780035f0 con 0x7eff841004f0 2024-04-25T15:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841a21a0 con 0x7eff841004f0 2024-04-25T15:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800ce60 con 0x7eff841004f0 2024-04-25T15:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.399+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841a26f0 con 0x7eff841004f0 2024-04-25T15:03:41.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.400+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8410c380 con 0x7eff841004f0 2024-04-25T15:03:41.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.402+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7eff78004000 con 0x7eff841004f0 2024-04-25T15:03:41.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.403+0000 7eff7ea00640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 0x7eff6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.404+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7eff780d0870 con 0x7eff841004f0 2024-04-25T15:03:41.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.404+0000 7eff89e00640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 0x7eff6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.404+0000 7eff89e00640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 0x7eff6c0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff70005fd0 tx=0x7eff7000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.408+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff7809d200 con 0x7eff841004f0 2024-04-25T15:03:41.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:41 smithi083 ceph-mon[32919]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:03:41.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:41 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3656606392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:41 smithi102 ceph-mon[33005]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:03:41.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:41 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3656606392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:41.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.614+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff84062b50 con 0x7eff841004f0 2024-04-25T15:03:41.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.615+0000 7eff7ea00640 1 -- 172.21.15.19:0/833075115 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7eff7809cb00 con 0x7eff841004f0 2024-04-25T15:03:41.617 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 msgr2=0x7eff6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 0x7eff6c0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff70005fd0 tx=0x7eff7000b480 comp rx=0 tx=0).stop 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 msgr2=0x7eff8419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9c0 tx=0x7eff7800ee80 comp rx=0 tx=0).stop 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 shutdown_connections 2024-04-25T15:03:41.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7eff6c0a3bb0 0x7eff6c0a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff84101060 0x7eff841a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff841004f0 0x7eff8419a7d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 --2- 172.21.15.19:0/833075115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff840ffb30 0x7eff8419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 >> 172.21.15.19:0/833075115 conn(0x7eff840fb610 msgr2=0x7eff840fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:41.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 shutdown_connections 2024-04-25T15:03:41.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.619+0000 7eff8b200640 1 -- 172.21.15.19:0/833075115 wait complete. 2024-04-25T15:03:41.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:41 smithi019 ceph-mon[28090]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:03:41.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:41 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3656606392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:41.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 -- 172.21.15.19:0/2988839100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c103040 msgr2=0x7ff54c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 --2- 172.21.15.19:0/2988839100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c103040 0x7ff54c1054b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff534009860 tx=0x7ff53402a340 comp rx=0 tx=0).stop 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 -- 172.21.15.19:0/2988839100 shutdown_connections 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 --2- 172.21.15.19:0/2988839100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c1059f0 0x7ff54c105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 --2- 172.21.15.19:0/2988839100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c103040 0x7ff54c1054b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 --2- 172.21.15.19:0/2988839100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 -- 172.21.15.19:0/2988839100 >> 172.21.15.19:0/2988839100 conn(0x7ff54c0fc2f0 msgr2=0x7ff54c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 -- 172.21.15.19:0/2988839100 shutdown_connections 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.861+0000 7ff551200640 1 -- 172.21.15.19:0/2988839100 wait complete. 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 Processor -- start 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 -- start start 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c1059f0 0x7ff54c1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.862+0000 7ff551200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c113f80 con 0x7ff54c1006d0 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff551200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c113e00 con 0x7ff54c1059f0 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff551200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c114100 con 0x7ff54c103040 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff542a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57630/0 (socket says 172.21.15.19:57630) 2024-04-25T15:03:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 -- 172.21.15.19:0/3507705146 learned_addr learned my addr 172.21.15.19:0/3507705146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff542a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:50896/0 (socket says 172.21.15.19:50896) 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54be00640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c1059f0 0x7ff54c1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 -- 172.21.15.19:0/3507705146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 msgr2=0x7ff54c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 -- 172.21.15.19:0/3507705146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c1059f0 msgr2=0x7ff54c1a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c1059f0 0x7ff54c1a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff54b400640 1 -- 172.21.15.19:0/3507705146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c1a6280 con 0x7ff54c1006d0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.863+0000 7ff542a00640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff54b400640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cde0 con 0x7ff54c1006d0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c0035f0 con 0x7ff54c1006d0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c1a6570 con 0x7ff54c1006d0 2024-04-25T15:03:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c010690 con 0x7ff54c1006d0 2024-04-25T15:03:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.864+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c1a6a90 con 0x7ff54c1006d0 2024-04-25T15:03:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.865+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54c0fdde0 con 0x7ff54c1006d0 2024-04-25T15:03:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.867+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff53c004000 con 0x7ff54c1006d0 2024-04-25T15:03:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.869+0000 7ff548c00640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 0x7ff5300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.869+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff53c0d0740 con 0x7ff54c1006d0 2024-04-25T15:03:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.869+0000 7ff542a00640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 0x7ff5300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.870+0000 7ff542a00640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 0x7ff5300a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff53402a930 tx=0x7ff534030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:41.874+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff53c009510 con 0x7ff54c1006d0 2024-04-25T15:03:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.079+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff54c062b50 con 0x7ff54c1006d0 2024-04-25T15:03:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.080+0000 7ff548c00640 1 -- 172.21.15.19:0/3507705146 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7ff53c09d0d0 con 0x7ff54c1006d0 2024-04-25T15:03:42.081 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:03:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 msgr2=0x7ff5300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 0x7ff5300a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff53402a930 tx=0x7ff534030450 comp rx=0 tx=0).stop 2024-04-25T15:03:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 msgr2=0x7ff54c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c00ee80 comp rx=0 tx=0).stop 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 shutdown_connections 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff5300a3bb0 0x7ff5300a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c1059f0 0x7ff54c1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff54c103040 0x7ff54c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 --2- 172.21.15.19:0/3507705146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff54c1006d0 0x7ff54c19e660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 >> 172.21.15.19:0/3507705146 conn(0x7ff54c0fc2f0 msgr2=0x7ff54c101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 shutdown_connections 2024-04-25T15:03:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.083+0000 7ff551200640 1 -- 172.21.15.19:0/3507705146 wait complete. 2024-04-25T15:03:42.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:03:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 -- 172.21.15.19:0/3812421584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54105280 msgr2=0x7fcf54105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 --2- 172.21.15.19:0/3812421584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54105280 0x7fcf54105700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009900 tx=0x7fcf3c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 -- 172.21.15.19:0/3812421584 shutdown_connections 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 --2- 172.21.15.19:0/3812421584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf5410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 --2- 172.21.15.19:0/3812421584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54105280 0x7fcf54105700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 --2- 172.21.15.19:0/3812421584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54104090 0x7fcf541044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 -- 172.21.15.19:0/3812421584 >> 172.21.15.19:0/3812421584 conn(0x7fcf540ff600 msgr2=0x7fcf54101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 -- 172.21.15.19:0/3812421584 shutdown_connections 2024-04-25T15:03:42.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.309+0000 7fcf59000640 1 -- 172.21.15.19:0/3812421584 wait complete. 2024-04-25T15:03:42.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.310+0000 7fcf59000640 1 Processor -- start 2024-04-25T15:03:42.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.310+0000 7fcf59000640 1 -- start start 2024-04-25T15:03:42.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 0x7fcf5419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54105280 0x7fcf5419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf541a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5410fc20 con 0x7fcf54105c40 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5410faa0 con 0x7fcf54105280 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf59000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5410fda0 con 0x7fcf54104090 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf541a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf53400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 0x7fcf5419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54105280 0x7fcf5419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf541a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57636/0 (socket says 172.21.15.19:57636) 2024-04-25T15:03:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.311+0000 7fcf53e00640 1 -- 172.21.15.19:0/2790005710 learned_addr learned my addr 172.21.15.19:0/2790005710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 -- 172.21.15.19:0/2790005710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54105280 msgr2=0x7fcf5419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54105280 0x7fcf5419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 -- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 msgr2=0x7fcf541a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf541a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 -- 172.21.15.19:0/2790005710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf541a1ed0 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf53400640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 0x7fcf5419a2b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf541050e0 tx=0x7fcf4400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.312+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44017070 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.313+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf541a21c0 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.313+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf44009c30 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.313+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4400f620 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.313+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf541a26e0 con 0x7fcf54104090 2024-04-25T15:03:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.314+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5410b3b0 con 0x7fcf54104090 2024-04-25T15:03:42.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.316+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcf4401a070 con 0x7fcf54104090 2024-04-25T15:03:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.317+0000 7fcf4be00640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 0x7fcf380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.317+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fcf440cfe10 con 0x7fcf54104090 2024-04-25T15:03:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.317+0000 7fcf52a00640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 0x7fcf380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.318+0000 7fcf52a00640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 0x7fcf380a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c031500 tx=0x7fcf3c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:42.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.322+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf440d3030 con 0x7fcf54104090 2024-04-25T15:03:42.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:42 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/833075115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:42 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3507705146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.529+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf54062b50 con 0x7fcf54104090 2024-04-25T15:03:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.530+0000 7fcf4be00640 1 -- 172.21.15.19:0/2790005710 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fcf4409c150 con 0x7fcf54104090 2024-04-25T15:03:42.531 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:03:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 msgr2=0x7fcf380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 0x7fcf380a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c031500 tx=0x7fcf3c005a20 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 msgr2=0x7fcf5419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 0x7fcf5419a2b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf541050e0 tx=0x7fcf4400dde0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 shutdown_connections 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf54105c40 0x7fcf541a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcf380a3bb0 0x7fcf380a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf54105280 0x7fcf5419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 --2- 172.21.15.19:0/2790005710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf54104090 0x7fcf5419a2b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 >> 172.21.15.19:0/2790005710 conn(0x7fcf540ff600 msgr2=0x7fcf54108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 shutdown_connections 2024-04-25T15:03:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.532+0000 7fcf59000640 1 -- 172.21.15.19:0/2790005710 wait complete. 2024-04-25T15:03:42.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:42 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/833075115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:42 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3507705146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:03:42.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:42 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/833075115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:42 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3507705146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 -- 172.21.15.19:0/916576624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 msgr2=0x7fc46c104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 --2- 172.21.15.19:0/916576624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c104c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 -- 172.21.15.19:0/916576624 shutdown_connections 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 --2- 172.21.15.19:0/916576624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 0x7fc46c109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 --2- 172.21.15.19:0/916576624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 0x7fc46c107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 --2- 172.21.15.19:0/916576624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c104c30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.763+0000 7fc470c00640 1 -- 172.21.15.19:0/916576624 >> 172.21.15.19:0/916576624 conn(0x7fc46c0fc230 msgr2=0x7fc46c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.764+0000 7fc470c00640 1 -- 172.21.15.19:0/916576624 shutdown_connections 2024-04-25T15:03:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.764+0000 7fc470c00640 1 -- 172.21.15.19:0/916576624 wait complete. 2024-04-25T15:03:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.765+0000 7fc470c00640 1 Processor -- start 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.765+0000 7fc470c00640 1 -- start start 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.765+0000 7fc470c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.765+0000 7fc470c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 0x7fc46c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.765+0000 7fc470c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 0x7fc46c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc470c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c10d940 con 0x7fc46c102840 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc470c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c10d7c0 con 0x7fc46c105170 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc470c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c10dac0 con 0x7fc46c107aa0 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57654/0 (socket says 172.21.15.19:57654) 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 -- 172.21.15.19:0/107690787 learned_addr learned my addr 172.21.15.19:0/107690787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46be00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 0x7fc46c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46aa00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 0x7fc46c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 -- 172.21.15.19:0/107690787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 msgr2=0x7fc46c1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 0x7fc46c1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 -- 172.21.15.19:0/107690787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 msgr2=0x7fc46c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 0x7fc46c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.766+0000 7fc46b400640 1 -- 172.21.15.19:0/107690787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1a1eb0 con 0x7fc46c102840 2024-04-25T15:03:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc46b400640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c03f070 con 0x7fc46c102840 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45c031c50 con 0x7fc46c102840 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c035030 con 0x7fc46c102840 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1a2140 con 0x7fc46c102840 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.767+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1a25e0 con 0x7fc46c102840 2024-04-25T15:03:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.768+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46c1095f0 con 0x7fc46c102840 2024-04-25T15:03:42.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.770+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc45c042070 con 0x7fc46c102840 2024-04-25T15:03:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.772+0000 7fc463e00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 0x7fc4500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.772+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fc45c0f0af0 con 0x7fc46c102840 2024-04-25T15:03:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.772+0000 7fc46aa00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 0x7fc4500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:42.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.773+0000 7fc46aa00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 0x7fc4500a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc46c19b7a0 tx=0x7fc458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:42.777+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc45c0bce30 con 0x7fc46c102840 2024-04-25T15:03:43.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.023+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46c02d5c0 con 0x7fc46c102840 2024-04-25T15:03:43.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.025+0000 7fc463e00640 1 -- 172.21.15.19:0/107690787 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7fc45c0c1130 con 0x7fc46c102840 2024-04-25T15:03:43.027 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:43.027 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:03:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:03:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 msgr2=0x7fc4500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 0x7fc4500a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc46c19b7a0 tx=0x7fc458005f90 comp rx=0 tx=0).stop 2024-04-25T15:03:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 msgr2=0x7fc46c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c003b90 comp rx=0 tx=0).stop 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 shutdown_connections 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc46c107aa0 0x7fc46c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc4500a3c30 0x7fc4500a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc46c105170 0x7fc46c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 --2- 172.21.15.19:0/107690787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc46c102840 0x7fc46c19a290 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 >> 172.21.15.19:0/107690787 conn(0x7fc46c0fc230 msgr2=0x7fc46c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 shutdown_connections 2024-04-25T15:03:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.028+0000 7fc470c00640 1 -- 172.21.15.19:0/107690787 wait complete. 2024-04-25T15:03:43.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 -- 172.21.15.19:0/2419533526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e38101ba0 msgr2=0x7f7e38108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 --2- 172.21.15.19:0/2419533526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e38101ba0 0x7f7e38108430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009800 tx=0x7f7e3002a580 comp rx=0 tx=0).stop 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 -- 172.21.15.19:0/2419533526 shutdown_connections 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 --2- 172.21.15.19:0/2419533526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e38101ba0 0x7f7e38108430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 --2- 172.21.15.19:0/2419533526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e38101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 --2- 172.21.15.19:0/2419533526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e380ffff0 0x7f7e38100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 -- 172.21.15.19:0/2419533526 >> 172.21.15.19:0/2419533526 conn(0x7f7e380fb580 msgr2=0x7f7e380fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 -- 172.21.15.19:0/2419533526 shutdown_connections 2024-04-25T15:03:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.257+0000 7f7e37e00640 1 -- 172.21.15.19:0/2419533526 wait complete. 2024-04-25T15:03:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.258+0000 7f7e37e00640 1 Processor -- start 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.258+0000 7f7e37e00640 1 -- start start 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e3819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38101ba0 0x7f7e381a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810b6f0 con 0x7f7e380ffff0 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810b570 con 0x7f7e381011e0 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810b870 con 0x7f7e38101ba0 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e36a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e36a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57666/0 (socket says 172.21.15.19:57666) 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e36a00640 1 -- 172.21.15.19:0/1925090872 learned_addr learned my addr 172.21.15.19:0/1925090872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e3819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.259+0000 7f7e37400640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38101ba0 0x7f7e381a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 -- 172.21.15.19:0/1925090872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38101ba0 msgr2=0x7f7e381a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38101ba0 0x7f7e381a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 -- 172.21.15.19:0/1925090872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 msgr2=0x7f7e3819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e3819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 -- 172.21.15.19:0/1925090872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e381a1dd0 con 0x7f7e380ffff0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e3819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e36a00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e9c0 tx=0x7f7e2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800cd80 con 0x7f7e380ffff0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e280035f0 con 0x7f7e380ffff0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28010690 con 0x7f7e380ffff0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e381a20c0 con 0x7f7e380ffff0 2024-04-25T15:03:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.260+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e381a2610 con 0x7f7e380ffff0 2024-04-25T15:03:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.261+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e38107310 con 0x7f7e380ffff0 2024-04-25T15:03:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.264+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e28004000 con 0x7f7e380ffff0 2024-04-25T15:03:43.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.266+0000 7f7e2f400640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 0x7f7e1c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.266+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f7e280d0960 con 0x7f7e380ffff0 2024-04-25T15:03:43.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.266+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 0x7f7e1c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.267+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 0x7f7e1c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e3819b6c0 tx=0x7f7e20010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:43.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.270+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e2809d2f0 con 0x7f7e380ffff0 2024-04-25T15:03:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:43 smithi019 ceph-mon[28090]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:43 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2790005710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:43.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:43 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/107690787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.479+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e38062b50 con 0x7f7e380ffff0 2024-04-25T15:03:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.484+0000 7f7e2f400640 1 -- 172.21.15.19:0/1925090872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7f7e2809cbf0 con 0x7f7e380ffff0 2024-04-25T15:03:43.485 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:43.485 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:03:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.486+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 msgr2=0x7f7e1c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.486+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 0x7f7e1c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e3819b6c0 tx=0x7f7e20010040 comp rx=0 tx=0).stop 2024-04-25T15:03:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.486+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 msgr2=0x7f7e3819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.486+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e9c0 tx=0x7f7e2800ee80 comp rx=0 tx=0).stop 2024-04-25T15:03:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 shutdown_connections 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38101ba0 0x7f7e381a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f7e1c0a3c30 0x7f7e1c0a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381011e0 0x7f7e3819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 --2- 172.21.15.19:0/1925090872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e380ffff0 0x7f7e3819a1b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 >> 172.21.15.19:0/1925090872 conn(0x7f7e380fb580 msgr2=0x7f7e381044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 shutdown_connections 2024-04-25T15:03:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.487+0000 7f7e37e00640 1 -- 172.21.15.19:0/1925090872 wait complete. 2024-04-25T15:03:43.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:03:43.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:03:43.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:43 smithi083 ceph-mon[32919]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:43.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:43 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2790005710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:43.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:43 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/107690787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:43.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:43 smithi102 ceph-mon[33005]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:43.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:43 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2790005710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:43.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:43 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/107690787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 -- 172.21.15.19:0/1301531456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc104090 msgr2=0x7fb6bc1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 --2- 172.21.15.19:0/1301531456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc104090 0x7fb6bc1044b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009900 tx=0x7fb6ac02a400 comp rx=0 tx=0).stop 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 -- 172.21.15.19:0/1301531456 shutdown_connections 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 --2- 172.21.15.19:0/1301531456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 --2- 172.21.15.19:0/1301531456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc105280 0x7fb6bc105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 --2- 172.21.15.19:0/1301531456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc104090 0x7fb6bc1044b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 -- 172.21.15.19:0/1301531456 >> 172.21.15.19:0/1301531456 conn(0x7fb6bc0ff600 msgr2=0x7fb6bc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 -- 172.21.15.19:0/1301531456 shutdown_connections 2024-04-25T15:03:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.714+0000 7fb6c1800640 1 -- 172.21.15.19:0/1301531456 wait complete. 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.715+0000 7fb6c1800640 1 Processor -- start 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.715+0000 7fb6c1800640 1 -- start start 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 0x7fb6bc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc10fc20 con 0x7fb6bc105280 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc10faa0 con 0x7fb6bc105c40 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc10fda0 con 0x7fb6bc104090 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 0x7fb6bc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57686/0 (socket says 172.21.15.19:57686) 2024-04-25T15:03:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6b3e00640 1 -- 172.21.15.19:0/3439833928 learned_addr learned my addr 172.21.15.19:0/3439833928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6c0e00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6b3e00640 1 -- 172.21.15.19:0/3439833928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 msgr2=0x7fb6bc19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.716+0000 7fb6b3e00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 0x7fb6bc19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b3e00640 1 -- 172.21.15.19:0/3439833928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 msgr2=0x7fb6bc1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b3e00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b3e00640 1 -- 172.21.15.19:0/3439833928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc1a1f00 con 0x7fb6bc105280 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6bbe00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 0x7fb6bc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6c0e00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b3e00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9f0 tx=0x7fb6a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ccc0 con 0x7fb6bc105280 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc1a21f0 con 0x7fb6bc105280 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a400ce60 con 0x7fb6bc105280 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4010430 con 0x7fb6bc105280 2024-04-25T15:03:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.717+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc1a26f0 con 0x7fb6bc105280 2024-04-25T15:03:43.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.718+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc10b430 con 0x7fb6bc105280 2024-04-25T15:03:43.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.721+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6a40046c0 con 0x7fb6bc105280 2024-04-25T15:03:43.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.722+0000 7fb6b9600640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 0x7fb6a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:43.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.722+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb6a40d0a00 con 0x7fb6bc105280 2024-04-25T15:03:43.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.722+0000 7fb6bbe00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 0x7fb6a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.723+0000 7fb6bbe00640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 0x7fb6a00a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac005ab0 tx=0x7fb6ac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.724+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6a4053ff0 con 0x7fb6bc105280 2024-04-25T15:03:43.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.931+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6bc02d810 con 0x7fb6bc105280 2024-04-25T15:03:43.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.932+0000 7fb6b9600640 1 -- 172.21.15.19:0/3439833928 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fb6a409d390 con 0x7fb6bc105280 2024-04-25T15:03:43.933 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 msgr2=0x7fb6a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 0x7fb6a00a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac005ab0 tx=0x7fb6ac005a20 comp rx=0 tx=0).stop 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 msgr2=0x7fb6bc19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9f0 tx=0x7fb6a400edf0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 shutdown_connections 2024-04-25T15:03:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fb6a00a3bb0 0x7fb6a00a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6bc105c40 0x7fb6bc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6bc105280 0x7fb6bc19a820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 --2- 172.21.15.19:0/3439833928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6bc104090 0x7fb6bc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.934+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 >> 172.21.15.19:0/3439833928 conn(0x7fb6bc0ff600 msgr2=0x7fb6bc108600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.935+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 shutdown_connections 2024-04-25T15:03:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:43.935+0000 7fb6c1800640 1 -- 172.21.15.19:0/3439833928 wait complete. 2024-04-25T15:03:43.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 -- 172.21.15.19:0/4195721230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d8108490 msgr2=0x7f51d81088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 --2- 172.21.15.19:0/4195721230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d8108490 0x7f51d81088b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009ca0 tx=0x7f51c802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 -- 172.21.15.19:0/4195721230 shutdown_connections 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 --2- 172.21.15.19:0/4195721230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d810a040 0x7f51d81108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 --2- 172.21.15.19:0/4195721230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 --2- 172.21.15.19:0/4195721230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d8108490 0x7f51d81088b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 -- 172.21.15.19:0/4195721230 >> 172.21.15.19:0/4195721230 conn(0x7f51d8077420 msgr2=0x7f51d8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 -- 172.21.15.19:0/4195721230 shutdown_connections 2024-04-25T15:03:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.170+0000 7f51d7200640 1 -- 172.21.15.19:0/4195721230 wait complete. 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.171+0000 7f51d7200640 1 Processor -- start 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 -- start start 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8108490 0x7f51d819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d810a040 0x7f51d81a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8114020 con 0x7f51d8109680 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8113ea0 con 0x7f51d8108490 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81141a0 con 0x7f51d810a040 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36910/0 (socket says 172.21.15.19:36910) 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d5400640 1 -- 172.21.15.19:0/1219502602 learned_addr learned my addr 172.21.15.19:0/1219502602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d5e00640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8108490 0x7f51d819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.172+0000 7f51d6800640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d810a040 0x7f51d81a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 -- 172.21.15.19:0/1219502602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d810a040 msgr2=0x7f51d81a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d810a040 0x7f51d81a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 -- 172.21.15.19:0/1219502602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8108490 msgr2=0x7f51d819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8108490 0x7f51d819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 -- 172.21.15.19:0/1219502602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d81a62c0 con 0x7f51d8109680 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d5400640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d950 tx=0x7f51c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51d8109680 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d81a65b0 con 0x7f51d8109680 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c0009c30 con 0x7f51d8109680 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.173+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000f620 con 0x7f51d8109680 2024-04-25T15:03:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.174+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d81a6b50 con 0x7f51d8109680 2024-04-25T15:03:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.175+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d810f830 con 0x7f51d8109680 2024-04-25T15:03:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.177+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51c0010460 con 0x7f51d8109680 2024-04-25T15:03:44.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.178+0000 7f51cea00640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 0x7f51bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.179+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f51c00cffd0 con 0x7f51d8109680 2024-04-25T15:03:44.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.179+0000 7f51d5e00640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 0x7f51bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.179+0000 7f51d5e00640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 0x7f51bc0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51c8000f80 tx=0x7f51c8005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:44.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.183+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51c0009de0 con 0x7f51d8109680 2024-04-25T15:03:44.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.397+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51d8062b50 con 0x7f51d8109680 2024-04-25T15:03:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.398+0000 7f51cea00640 1 -- 172.21.15.19:0/1219502602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f51c009c310 con 0x7f51d8109680 2024-04-25T15:03:44.399 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:03:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 msgr2=0x7f51bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 0x7f51bc0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51c8000f80 tx=0x7f51c8005f30 comp rx=0 tx=0).stop 2024-04-25T15:03:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 msgr2=0x7f51d819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d950 tx=0x7f51c000de10 comp rx=0 tx=0).stop 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 shutdown_connections 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d810a040 0x7f51d81a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d8109680 0x7f51d819ebe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f51bc0a3bb0 0x7f51bc0a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 --2- 172.21.15.19:0/1219502602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8108490 0x7f51d819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 >> 172.21.15.19:0/1219502602 conn(0x7f51d8077420 msgr2=0x7f51d810ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 shutdown_connections 2024-04-25T15:03:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.401+0000 7f51d7200640 1 -- 172.21.15.19:0/1219502602 wait complete. 2024-04-25T15:03:44.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:03:44.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:44 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1925090872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:44.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:44 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3439833928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:44.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:44 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1925090872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:44.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:44 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3439833928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:44.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1925090872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:44.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:44 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3439833928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 -- 172.21.15.19:0/4170389288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 msgr2=0x7f2c201088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 --2- 172.21.15.19:0/4170389288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c201088b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c1002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 -- 172.21.15.19:0/4170389288 shutdown_connections 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 --2- 172.21.15.19:0/4170389288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 0x7f2c201108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 --2- 172.21.15.19:0/4170389288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 0x7f2c20109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 --2- 172.21.15.19:0/4170389288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c201088b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 -- 172.21.15.19:0/4170389288 >> 172.21.15.19:0/4170389288 conn(0x7f2c20077420 msgr2=0x7f2c20079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 -- 172.21.15.19:0/4170389288 shutdown_connections 2024-04-25T15:03:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.644+0000 7f2c1f200640 1 -- 172.21.15.19:0/4170389288 wait complete. 2024-04-25T15:03:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.645+0000 7f2c1f200640 1 Processor -- start 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 -- start start 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 0x7f2c2019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 0x7f2c201a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c20114020 con 0x7f2c20108490 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c20113ea0 con 0x7f2c20109680 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201141a0 con 0x7f2c2010a040 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36922/0 (socket says 172.21.15.19:36922) 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1de00640 1 -- 172.21.15.19:0/700716779 learned_addr learned my addr 172.21.15.19:0/700716779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c17e00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 0x7f2c2019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.646+0000 7f2c1e800640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 0x7f2c201a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 -- 172.21.15.19:0/700716779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 msgr2=0x7f2c201a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 0x7f2c201a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 -- 172.21.15.19:0/700716779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 msgr2=0x7f2c2019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 0x7f2c2019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 -- 172.21.15.19:0/700716779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c201a62d0 con 0x7f2c20108490 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1de00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c10031500 tx=0x7f2c10003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1003f070 con 0x7f2c20108490 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c10031c50 con 0x7f2c20108490 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10035030 con 0x7f2c20108490 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c201a6560 con 0x7f2c20108490 2024-04-25T15:03:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.647+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201a6aa0 con 0x7f2c20108490 2024-04-25T15:03:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.649+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2010f830 con 0x7f2c20108490 2024-04-25T15:03:44.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.651+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c10042070 con 0x7f2c20108490 2024-04-25T15:03:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.652+0000 7f2c16a00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 0x7f2c040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.652+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2c100f1100 con 0x7f2c20108490 2024-04-25T15:03:44.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.652+0000 7f2c17e00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 0x7f2c040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:44.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.653+0000 7f2c17e00640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 0x7f2c040a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c2019fbc0 tx=0x7f2c08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.654+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c100bd4a0 con 0x7f2c20108490 2024-04-25T15:03:44.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.864+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c20062b50 con 0x7f2c20108490 2024-04-25T15:03:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.865+0000 7f2c16a00640 1 -- 172.21.15.19:0/700716779 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f2c100c17a0 con 0x7f2c20108490 2024-04-25T15:03:44.866 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 msgr2=0x7f2c040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 0x7f2c040a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c2019fbc0 tx=0x7f2c08005f90 comp rx=0 tx=0).stop 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 msgr2=0x7f2c2019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c10031500 tx=0x7f2c10003c10 comp rx=0 tx=0).stop 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 shutdown_connections 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c2010a040 0x7f2c201a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f2c040a3c30 0x7f2c040a60f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c20109680 0x7f2c2019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 --2- 172.21.15.19:0/700716779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c20108490 0x7f2c2019e6b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:44.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 >> 172.21.15.19:0/700716779 conn(0x7f2c20077420 msgr2=0x7f2c2010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:44.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 shutdown_connections 2024-04-25T15:03:44.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:44.868+0000 7f2c1f200640 1 -- 172.21.15.19:0/700716779 wait complete. 2024-04-25T15:03:44.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 -- 172.21.15.19:0/455540382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81013c0 msgr2=0x7fa3b8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/455540382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81013c0 0x7fa3b8101840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009900 tx=0x7fa3a402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 -- 172.21.15.19:0/455540382 shutdown_connections 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/455540382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b8108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/455540382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81013c0 0x7fa3b8101840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/455540382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81001d0 0x7fa3b81005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 -- 172.21.15.19:0/455540382 >> 172.21.15.19:0/455540382 conn(0x7fa3b80fb740 msgr2=0x7fa3b80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 -- 172.21.15.19:0/455540382 shutdown_connections 2024-04-25T15:03:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.090+0000 7fa3bfe00640 1 -- 172.21.15.19:0/455540382 wait complete. 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.091+0000 7fa3bfe00640 1 Processor -- start 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 -- start start 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81013c0 0x7fa3b8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b810b8d0 con 0x7fa3b81001d0 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b810b750 con 0x7fa3b81013c0 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b810ba50 con 0x7fa3b8101d80 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36944/0 (socket says 172.21.15.19:36944) 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bd400640 1 -- 172.21.15.19:0/2820742581 learned_addr learned my addr 172.21.15.19:0/2820742581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bde00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.092+0000 7fa3bca00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81013c0 0x7fa3b8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 -- 172.21.15.19:0/2820742581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 msgr2=0x7fa3b819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 -- 172.21.15.19:0/2820742581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81013c0 msgr2=0x7fa3b8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81013c0 0x7fa3b8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 -- 172.21.15.19:0/2820742581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b819fde0 con 0x7fa3b81001d0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bde00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bd400640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00e9c0 tx=0x7fa3ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00cd80 con 0x7fa3b81001d0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ac0035f0 con 0x7fa3b81001d0 2024-04-25T15:03:45.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b81a00d0 con 0x7fa3b81001d0 2024-04-25T15:03:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac010690 con 0x7fa3b81001d0 2024-04-25T15:03:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.093+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b81a0640 con 0x7fa3b81001d0 2024-04-25T15:03:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.094+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b81a0900 con 0x7fa3b81001d0 2024-04-25T15:03:45.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.097+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3ac0037a0 con 0x7fa3b81001d0 2024-04-25T15:03:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.098+0000 7fa3b6000640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 0x7fa3a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.098+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fa3ac0d0730 con 0x7fa3b81001d0 2024-04-25T15:03:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.098+0000 7fa3bca00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 0x7fa3a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.099+0000 7fa3bca00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 0x7fa3a00a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.102+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3ac09d0c0 con 0x7fa3b81001d0 2024-04-25T15:03:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.358+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3b81a1130 con 0x7fa3b81001d0 2024-04-25T15:03:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.360+0000 7fa3b6000640 1 -- 172.21.15.19:0/2820742581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7fa3ac09c9c0 con 0x7fa3b81001d0 2024-04-25T15:03:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:03:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:03:45.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.362+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 msgr2=0x7fa3a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.362+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 0x7fa3a00a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a40314a0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.362+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 msgr2=0x7fa3b81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.362+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00e9c0 tx=0x7fa3ac00ee80 comp rx=0 tx=0).stop 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 shutdown_connections 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3b8101d80 0x7fa3b819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa3a00a3c30 0x7fa3a00a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3b81013c0 0x7fa3b8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 --2- 172.21.15.19:0/2820742581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b81001d0 0x7fa3b81981c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 >> 172.21.15.19:0/2820742581 conn(0x7fa3b80fb740 msgr2=0x7fa3b81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 shutdown_connections 2024-04-25T15:03:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.363+0000 7fa3bfe00640 1 -- 172.21.15.19:0/2820742581 wait complete. 2024-04-25T15:03:45.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:03:45.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:45 smithi083 ceph-mon[32919]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:45.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:45 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1219502602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:45 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/700716779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:45 smithi102 ceph-mon[33005]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:45.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:45 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1219502602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:45 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/700716779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 -- 172.21.15.19:0/1702924694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 msgr2=0x7fdf201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 --2- 172.21.15.19:0/1702924694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf201043f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009900 tx=0x7fdf1402a400 comp rx=0 tx=0).stop 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 -- 172.21.15.19:0/1702924694 shutdown_connections 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 --2- 172.21.15.19:0/1702924694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 0x7fdf2010c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 --2- 172.21.15.19:0/1702924694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf20105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 --2- 172.21.15.19:0/1702924694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf201043f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 -- 172.21.15.19:0/1702924694 >> 172.21.15.19:0/1702924694 conn(0x7fdf200ff560 msgr2=0x7fdf20101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 -- 172.21.15.19:0/1702924694 shutdown_connections 2024-04-25T15:03:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.588+0000 7fdf27200640 1 -- 172.21.15.19:0/1702924694 wait complete. 2024-04-25T15:03:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 Processor -- start 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 -- start start 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf2019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 0x7fdf201a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010fb60 con 0x7fdf201051c0 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010f9e0 con 0x7fdf20103fd0 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.589+0000 7fdf27200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010fce0 con 0x7fdf20105b80 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36972/0 (socket says 172.21.15.19:36972) 2024-04-25T15:03:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 -- 172.21.15.19:0/1658050282 learned_addr learned my addr 172.21.15.19:0/1658050282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25e00640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf2019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf26800640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 0x7fdf201a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 -- 172.21.15.19:0/1658050282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 msgr2=0x7fdf201a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 0x7fdf201a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 -- 172.21.15.19:0/1658050282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 msgr2=0x7fdf2019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf2019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 -- 172.21.15.19:0/1658050282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a1e30 con 0x7fdf201051c0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.590+0000 7fdf25400640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9f0 tx=0x7fdf1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.591+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000cd80 con 0x7fdf201051c0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.591+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf100108e0 con 0x7fdf201051c0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.591+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a2120 con 0x7fdf201051c0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.591+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10018980 con 0x7fdf201051c0 2024-04-25T15:03:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.591+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a2670 con 0x7fdf201051c0 2024-04-25T15:03:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.592+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2010b370 con 0x7fdf201051c0 2024-04-25T15:03:45.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.595+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf10010430 con 0x7fdf201051c0 2024-04-25T15:03:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.596+0000 7fdf1aa00640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 0x7fdf080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.596+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fdf100d0720 con 0x7fdf201051c0 2024-04-25T15:03:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.596+0000 7fdf25e00640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 0x7fdf080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:45.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.597+0000 7fdf25e00640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 0x7fdf080a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005ab0 tx=0x7fdf14005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:45.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.600+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf1009ca60 con 0x7fdf201051c0 2024-04-25T15:03:45.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:45 smithi019 ceph-mon[28090]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:45.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:45 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1219502602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:45 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/700716779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:03:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.807+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf20062b50 con 0x7fdf201051c0 2024-04-25T15:03:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.810+0000 7fdf1aa00640 1 -- 172.21.15.19:0/1658050282 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7fdf100a0d60 con 0x7fdf201051c0 2024-04-25T15:03:45.812 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:45.812 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:03:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 msgr2=0x7fdf080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 0x7fdf080a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005ab0 tx=0x7fdf14005a20 comp rx=0 tx=0).stop 2024-04-25T15:03:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 msgr2=0x7fdf2019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9f0 tx=0x7fdf1000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 shutdown_connections 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf20105b80 0x7fdf201a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf201051c0 0x7fdf2019a750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fdf080a3bb0 0x7fdf080a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 --2- 172.21.15.19:0/1658050282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf20103fd0 0x7fdf2019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.813+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 >> 172.21.15.19:0/1658050282 conn(0x7fdf200ff560 msgr2=0x7fdf20108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.814+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 shutdown_connections 2024-04-25T15:03:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:45.814+0000 7fdf27200640 1 -- 172.21.15.19:0/1658050282 wait complete. 2024-04-25T15:03:45.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:03:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 -- 172.21.15.19:0/2747185400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b8109f80 msgr2=0x7f41b8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 --2- 172.21.15.19:0/2747185400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b8109f80 0x7f41b8110810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009be0 tx=0x7f41b002a6c0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 -- 172.21.15.19:0/2747185400 shutdown_connections 2024-04-25T15:03:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 --2- 172.21.15.19:0/2747185400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b8109f80 0x7f41b8110810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 --2- 172.21.15.19:0/2747185400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b81095c0 0x7f41b8109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 --2- 172.21.15.19:0/2747185400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 0x7f41b81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 -- 172.21.15.19:0/2747185400 >> 172.21.15.19:0/2747185400 conn(0x7f41b8077390 msgr2=0x7f41b80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 -- 172.21.15.19:0/2747185400 shutdown_connections 2024-04-25T15:03:46.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.052+0000 7f41bd800640 1 -- 172.21.15.19:0/2747185400 wait complete. 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.053+0000 7f41bd800640 1 Processor -- start 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 -- start start 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 0x7f41b819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109f80 0x7f41b81a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113f10 con 0x7f41b81083d0 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113d90 con 0x7f41b81095c0 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41bd800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8114090 con 0x7f41b8109f80 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 0x7f41b819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41afe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.054+0000 7f41afe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40354/0 (socket says 172.21.15.19:40354) 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 -- 172.21.15.19:0/2211210801 learned_addr learned my addr 172.21.15.19:0/2211210801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41bce00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109f80 0x7f41b81a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 -- 172.21.15.19:0/2211210801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109f80 msgr2=0x7f41b81a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109f80 0x7f41b81a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 -- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 msgr2=0x7f41b819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 0x7f41b819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 -- 172.21.15.19:0/2211210801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b81a61c0 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41afe00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.055+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000cd80 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.056+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a00054d0 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.056+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b81a64b0 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.056+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0010690 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.056+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a69b0 con 0x7f41b81095c0 2024-04-25T15:03:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.057+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b810f770 con 0x7f41b81095c0 2024-04-25T15:03:46.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.059+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f41a0004e90 con 0x7f41b81095c0 2024-04-25T15:03:46.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.060+0000 7f41b5600640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 0x7f419c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.060+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f41a00d0820 con 0x7f41b81095c0 2024-04-25T15:03:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.061+0000 7f41b7e00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 0x7f419c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.061+0000 7f41b7e00640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 0x7f419c0a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009530 tx=0x7f41a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.063+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41a009d1b0 con 0x7f41b81095c0 2024-04-25T15:03:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.310+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41b802d810 con 0x7f41b81095c0 2024-04-25T15:03:46.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.312+0000 7f41b5600640 1 -- 172.21.15.19:0/2211210801 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7f41a009cab0 con 0x7f41b81095c0 2024-04-25T15:03:46.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:03:46.314 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:03:46.314 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:03:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.314+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 msgr2=0x7f419c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.314+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 0x7f419c0a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009530 tx=0x7f41a8009260 comp rx=0 tx=0).stop 2024-04-25T15:03:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.314+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 msgr2=0x7f41b819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.314+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000edf0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 shutdown_connections 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109f80 0x7f41b81a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f419c0a3bb0 0x7f419c0a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41b81095c0 0x7f41b819eae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 --2- 172.21.15.19:0/2211210801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81083d0 0x7f41b819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 >> 172.21.15.19:0/2211210801 conn(0x7f41b8077390 msgr2=0x7f41b810c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 shutdown_connections 2024-04-25T15:03:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.315+0000 7f41bd800640 1 -- 172.21.15.19:0/2211210801 wait complete. 2024-04-25T15:03:46.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T15:02:00.518771+0000', 'modified': '2024-04-25T15:02:04.773178+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24590}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/2613932433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 2613932433}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 2613932433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24590': {'gid': 24590, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/2949462770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2949462770}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2949462770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24590, 'qdb_cluster': [24590]} 2024-04-25T15:03:46.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T15:03:46.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T15:03:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 -- 172.21.15.19:0/4010197254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9640749b0 msgr2=0x7ff964074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 --2- 172.21.15.19:0/4010197254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9640749b0 0x7ff964074e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff950009900 tx=0x7ff95002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 -- 172.21.15.19:0/4010197254 shutdown_connections 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 --2- 172.21.15.19:0/4010197254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964109d20 0x7ff96410c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 --2- 172.21.15.19:0/4010197254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9640749b0 0x7ff964074e30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 --2- 172.21.15.19:0/4010197254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff964075f90 0x7ff9640743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 -- 172.21.15.19:0/4010197254 >> 172.21.15.19:0/4010197254 conn(0x7ff9640fd3f0 msgr2=0x7ff9640ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 -- 172.21.15.19:0/4010197254 shutdown_connections 2024-04-25T15:03:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.539+0000 7ff969000640 1 -- 172.21.15.19:0/4010197254 wait complete. 2024-04-25T15:03:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.540+0000 7ff969000640 1 Processor -- start 2024-04-25T15:03:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.540+0000 7ff969000640 1 -- start start 2024-04-25T15:03:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9640749b0 0x7ff96419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964075f90 0x7ff96419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410fb50 con 0x7ff964109d20 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410f9d0 con 0x7ff9640749b0 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff969000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410fcd0 con 0x7ff964075f90 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff962a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964075f90 0x7ff96419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36996/0 (socket says 172.21.15.19:36996) 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 -- 172.21.15.19:0/2329208760 learned_addr learned my addr 172.21.15.19:0/2329208760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963400640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9640749b0 0x7ff96419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 -- 172.21.15.19:0/2329208760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964075f90 msgr2=0x7ff96419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964075f90 0x7ff96419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 -- 172.21.15.19:0/2329208760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9640749b0 msgr2=0x7ff96419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9640749b0 0x7ff96419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.541+0000 7ff963e00640 1 -- 172.21.15.19:0/2329208760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9641a1ea0 con 0x7ff964109d20 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff963e00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e970 tx=0x7ff95c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff95be00640 1 -- 172.21.15.19:0/2329208760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00ccc0 con 0x7ff964109d20 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff95be00640 1 -- 172.21.15.19:0/2329208760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c00ce60 con 0x7ff964109d20 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff95be00640 1 -- 172.21.15.19:0/2329208760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c010430 con 0x7ff964109d20 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641a2190 con 0x7ff964109d20 2024-04-25T15:03:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.542+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a2660 con 0x7ff964109d20 2024-04-25T15:03:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.545+0000 7ff96d5da740 1 Processor -- start 2024-04-25T15:03:46.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.545+0000 7ff95be00640 1 -- 172.21.15.19:0/2329208760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff95c004ff0 con 0x7ff964109d20 2024-04-25T15:03:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 -- start start 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 0x5646f1a79570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a7e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a7ecb0 0x5646f1a81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646f19ddb10 con 0x5646f1a7ecb0 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f19842b0 con 0x5646f1a79150 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff96d5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646f19d9ad0 con 0x5646f1a79b40 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff962a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 0x5646f1a79570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff962a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 0x5646f1a79570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:40374/0 (socket says 172.21.15.19:40374) 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff962a00640 1 -- 172.21.15.19:0/231571395 learned_addr learned my addr 172.21.15.19:0/231571395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff963400640 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a7e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.546+0000 7ff962a00640 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a7ecb0 0x5646f1a81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 -- 172.21.15.19:0/231571395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 msgr2=0x5646f1a79570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 0x5646f1a79570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 -- 172.21.15.19:0/231571395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a7ecb0 msgr2=0x5646f1a81140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a7ecb0 0x5646f1a81140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 -- 172.21.15.19:0/231571395 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f16f02d0 con 0x5646f1a79b40 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff95be00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 0x7ff9480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff963400640 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a7e640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff954009fa0 tx=0x7ff9540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1900549468b8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.547+0000 7ff95be00640 1 -- 172.21.15.19:0/2329208760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff95c0d07c0 con 0x7ff964109d20 2024-04-25T15:03:46.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff962a00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 0x7ff9480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff958c00640 1 -- 172.21.15.19:0/231571395 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954003c30 con 0x5646f1a79b40 2024-04-25T15:03:46.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff958c00640 1 -- 172.21.15.19:0/231571395 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff954003dd0 con 0x5646f1a79b40 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff958c00640 1 -- 172.21.15.19:0/231571395 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95400fa70 con 0x5646f1a79b40 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff96d5da740 1 -- 172.21.15.19:0/231571395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 msgr2=0x5646f1a7e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff962a00640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 0x7ff9480a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5646f1a7a3a0 tx=0x7ff95002a3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.548+0000 7ff96d5da740 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a7e640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff954009fa0 tx=0x7ff9540118d0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 -- 172.21.15.19:0/231571395 shutdown_connections 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a7ecb0 0x5646f1a81140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a7e640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 --2- 172.21.15.19:0/231571395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79150 0x5646f1a79570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 -- 172.21.15.19:0/231571395 >> 172.21.15.19:0/231571395 conn(0x5646f1a747c0 msgr2=0x5646f1a74bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 -- 172.21.15.19:0/231571395 shutdown_connections 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 -- 172.21.15.19:0/231571395 wait complete. 2024-04-25T15:03:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 -- start start 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.549+0000 7ff96d5da740 1 Processor -- start 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a83880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a81c60 con 0x5646f1a79150 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f1997570 con 0x5646f1a7ecb0 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a81de0 con 0x5646f1a79b40 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff96d5da740 10 client.0 _ll_register_callbacks cb 0x5646f1965fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37002/0 (socket says 172.21.15.19:37002) 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 -- 172.21.15.19:0/2188109019 learned_addr learned my addr 172.21.15.19:0/2188109019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963e00640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff962a00640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a83880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 -- 172.21.15.19:0/2188109019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 msgr2=0x5646f1a83880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a83880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 -- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a7ecb0 msgr2=0x5646f1a8a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.550+0000 7ff963400640 1 -- 172.21.15.19:0/2188109019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f17ba6b0 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff963400640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff95401a000 tx=0x7ff954004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e440cc79237711b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff94d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954028020 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff95400fa00 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff954003dd0 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff95400fc40 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff96d5da740 10 client.14874 fetch_fsmap learned FSMap version 20 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646f1a8dc30 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.551+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff954003dd0 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff954005040 con 0x5646f1a79150 2024-04-25T15:03:46.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 10 client.14874 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 10 client.14874 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 20 client.14874 populate_metadata read hostname 'smithi019' 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1a90060 0x5646f1a914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 4 client.14874 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f19eb530 con 0x5646f1a90060 2024-04-25T15:03:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.552+0000 7ff962a00640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1a90060 0x5646f1a914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2820742581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:46.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1658050282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.553+0000 7ff962a00640 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1a90060 0x5646f1a914d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d722962ce2dbdd2c server_cookie=67fa8054be1608d3 in_seq=0 out_seq=0 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.553+0000 7ff94d600640 10 client.14874 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff94d600640 1 -- 172.21.15.19:0/2188109019 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646f19eb530 con 0x5646f1a90060 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff94d600640 10 client.14874 handle_command_reply: tid=0 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 client.14874 shutdown 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 msgr2=0x5646f1a83340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff95401a000 tx=0x7ff954004cd0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 shutdown_connections 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1a90060 0x5646f1a914d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a79b40 0x5646f1a83880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 --2- 172.21.15.19:0/2188109019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a83340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff94d600640 0 client.14874 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 >> 172.21.15.19:0/2188109019 conn(0x5646f1a747c0 msgr2=0x5646f1a73a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 shutdown_connections 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 1 -- 172.21.15.19:0/2188109019 wait complete. 2024-04-25T15:03:46.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.554+0000 7ff96d5da740 20 client.14874 trim_cache size 0 max 16384 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 Processor -- start 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 -- start start 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a6f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a8a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a8f810 con 0x5646f1a79150 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a8fee0 con 0x5646f1a79b40 2024-04-25T15:03:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff96d5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a905b0 con 0x5646f1a7ecb0 2024-04-25T15:03:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a6f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a6f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37006/0 (socket says 172.21.15.19:37006) 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff963e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff963400640 1 -- 172.21.15.19:0/3840006645 learned_addr learned my addr 172.21.15.19:0/3840006645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.574+0000 7ff962a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a8a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 -- 172.21.15.19:0/3840006645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 msgr2=0x5646f1a8a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a8a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 -- 172.21.15.19:0/3840006645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 msgr2=0x5646f1a6f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a6f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 -- 172.21.15.19:0/3840006645 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f1a84100 con 0x5646f1a7ecb0 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff962a00640 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff963e00640 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8f040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e940 tx=0x7ff95c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e49321eb5e9e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff94fe00640 1 -- 172.21.15.19:0/3840006645 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0bd0b0 con 0x5646f1a7ecb0 2024-04-25T15:03:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff94fe00640 1 -- 172.21.15.19:0/3840006645 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c09d150 con 0x5646f1a7ecb0 2024-04-25T15:03:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.575+0000 7ff94fe00640 1 -- 172.21.15.19:0/3840006645 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c09ade0 con 0x5646f1a7ecb0 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 -- 172.21.15.19:0/3840006645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 msgr2=0x5646f1a8f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8f040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e940 tx=0x7ff95c0bd040 comp rx=0 tx=0).stop 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 -- 172.21.15.19:0/3840006645 shutdown_connections 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a8f040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 --2- 172.21.15.19:0/3840006645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a6f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 -- 172.21.15.19:0/3840006645 >> 172.21.15.19:0/3840006645 conn(0x5646f1a747c0 msgr2=0x5646f1a74ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 -- 172.21.15.19:0/3840006645 shutdown_connections 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.576+0000 7ff96d5da740 1 -- 172.21.15.19:0/3840006645 wait complete. 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 -- start start 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 Processor -- start 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a91350 con 0x5646f1a79150 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a90c80 con 0x5646f1a79b40 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646f1a91a20 con 0x5646f1a7ecb0 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff96d5da740 10 client.0 _ll_register_callbacks cb 0x7ff954009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:03:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.577+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37008/0 (socket says 172.21.15.19:37008) 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 -- 172.21.15.19:0/4207946136 learned_addr learned my addr 172.21.15.19:0/4207946136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963e00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff962a00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 -- 172.21.15.19:0/4207946136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 msgr2=0x5646f1a88fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a88fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 -- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 msgr2=0x5646f1a85fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a85fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 -- 172.21.15.19:0/4207946136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f1a895a0 con 0x5646f1a79150 2024-04-25T15:03:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963e00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff962a00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff963400640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff954009a00 tx=0x7ff954005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878238e44ec37472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff94d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.578+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954028040 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff95400f560 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c09d150 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff954003dc0 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 10 client.14880 fetch_fsmap learned FSMap version 20 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646f1a8a8a0 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c09d150 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff954040030 con 0x5646f1a79150 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 10 client.14880 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:03:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 10 client.14880 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 20 client.14880 populate_metadata read hostname 'smithi019' 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1afa7c0 0x5646f1af94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 4 client.14880 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.579+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "status"}) -- 0x5646f19e53a0 con 0x5646f1afa7c0 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.580+0000 7ff962a00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1afa7c0 0x5646f1af94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.580+0000 7ff962a00640 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1afa7c0 0x5646f1af94d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973a7514a0d1bc17 server_cookie=de535286675422eb in_seq=0 out_seq=0 2024-04-25T15:03:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.580+0000 7ff94d600640 10 client.14880 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff94d600640 1 -- 172.21.15.19:0/4207946136 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5646f19e53a0 con 0x5646f1afa7c0 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff94d600640 10 client.14880 handle_command_reply: tid=0 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 client.14880 shutdown 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 msgr2=0x5646f1a85aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff954009a00 tx=0x7ff954005b90 comp rx=0 tx=0).stop 2024-04-25T15:03:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 shutdown_connections 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x5646f1afa7c0 0x5646f1af94d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646f1a7ecb0 0x5646f1a88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646f1a79b40 0x5646f1a85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 --2- 172.21.15.19:0/4207946136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646f1a79150 0x5646f1a85aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff94d600640 0 client.14880 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 >> 172.21.15.19:0/4207946136 conn(0x5646f1a747c0 msgr2=0x5646f1a6f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 shutdown_connections 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 1 -- 172.21.15.19:0/4207946136 wait complete. 2024-04-25T15:03:46.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.581+0000 7ff96d5da740 20 client.14880 trim_cache size 0 max 16384 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24590, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 105.535748778, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 20, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 90, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 113.164308588 2024-04-25T15:03:46.588 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:03:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 msgr2=0x7ff9480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 0x7ff9480a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5646f1a7a3a0 tx=0x7ff95002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 msgr2=0x7ff9641a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e970 tx=0x7ff95c00ed70 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 shutdown_connections 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff964109d20 0x7ff9641a1830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff964075f90 0x7ff96419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7ff9480a3c30 0x7ff9480a60f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 --2- 172.21.15.19:0/2329208760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9640749b0 0x7ff96419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 >> 172.21.15.19:0/2329208760 conn(0x7ff9640fd3f0 msgr2=0x7ff964105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 shutdown_connections 2024-04-25T15:03:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:46.582+0000 7ff969000640 1 -- 172.21.15.19:0/2329208760 wait complete. 2024-04-25T15:03:46.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2820742581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:46.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1658050282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:46.600 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T15:03:46.600 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:03:46.609 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:03:46.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T15:03:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2820742581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1658050282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:03:47.039 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:47.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:47 smithi019 ceph-mon[28090]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:47.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:47 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2211210801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:47 smithi102 ceph-mon[33005]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:47.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:47 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2211210801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:47.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:47 smithi083 ceph-mon[32919]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:47.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:47 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2211210801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:03:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.507+0000 7fbd38a00700 1 -- 172.21.15.19:0/968599898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 msgr2=0x7fbd340758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.507+0000 7fbd38a00700 1 --2- 172.21.15.19:0/968599898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd340758a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009990 tx=0x7fbd20009ca0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 -- 172.21.15.19:0/968599898 shutdown_connections 2024-04-25T15:03:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 --2- 172.21.15.19:0/968599898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 0x7fbd3410cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 --2- 172.21.15.19:0/968599898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 0x7fbd34074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 --2- 172.21.15.19:0/968599898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd340758a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 -- 172.21.15.19:0/968599898 >> 172.21.15.19:0/968599898 conn(0x7fbd340fdf80 msgr2=0x7fbd341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 -- 172.21.15.19:0/968599898 shutdown_connections 2024-04-25T15:03:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.508+0000 7fbd38a00700 1 -- 172.21.15.19:0/968599898 wait complete. 2024-04-25T15:03:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 Processor -- start 2024-04-25T15:03:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 -- start start 2024-04-25T15:03:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 0x7fbd3419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 0x7fbd341a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34110620 con 0x7fbd34075420 2024-04-25T15:03:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341104a0 con 0x7fbd3410a8c0 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd38a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341107a0 con 0x7fbd340769f0 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.509+0000 7fbd33400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37026/0 (socket says 172.21.15.19:37026) 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 -- 172.21.15.19:0/490355800 learned_addr learned my addr 172.21.15.19:0/490355800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33e00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 0x7fbd341a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd32a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 0x7fbd3419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 -- 172.21.15.19:0/490355800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 msgr2=0x7fbd3419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 0x7fbd3419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 -- 172.21.15.19:0/490355800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 msgr2=0x7fbd341a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 0x7fbd341a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 -- 172.21.15.19:0/490355800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd341a2920 con 0x7fbd34075420 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.510+0000 7fbd33400700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9a0 tx=0x7fbd2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.511+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400cbd0 con 0x7fbd34075420 2024-04-25T15:03:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.511+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd2400cd70 con 0x7fbd34075420 2024-04-25T15:03:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.511+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24018680 con 0x7fbd34075420 2024-04-25T15:03:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.511+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd341a2c10 con 0x7fbd34075420 2024-04-25T15:03:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.511+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd341a31a0 con 0x7fbd34075420 2024-04-25T15:03:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.512+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34069730 con 0x7fbd34075420 2024-04-25T15:03:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.514+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbd24010430 con 0x7fbd34075420 2024-04-25T15:03:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.516+0000 7fbd2be00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 0x7fbd180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.516+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbd240d0a10 con 0x7fbd34075420 2024-04-25T15:03:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.516+0000 7fbd32a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 0x7fbd180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.517+0000 7fbd32a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 0x7fbd180a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b2e0 tx=0x7fbd2001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.519+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd2409d400 con 0x7fbd34075420 2024-04-25T15:03:49.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:49 smithi102 ceph-mon[33005]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:49.562 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:49 smithi019 ceph-mon[28090]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:49.593 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:49 smithi083 ceph-mon[32919]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:49.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.765+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fbd34062e80 con 0x7fbd180a3cc0 2024-04-25T15:03:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.767+0000 7fbd2be00700 1 -- 172.21.15.19:0/490355800 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fbd34062e80 con 0x7fbd180a3cc0 2024-04-25T15:03:49.768 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-04-25T15:03:49.768 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-04-25T15:03:49.769 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-04-25T15:03:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 msgr2=0x7fbd180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 0x7fbd180a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b2e0 tx=0x7fbd2001a040 comp rx=0 tx=0).stop 2024-04-25T15:03:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 msgr2=0x7fbd3419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400e9a0 tx=0x7fbd2400ed60 comp rx=0 tx=0).stop 2024-04-25T15:03:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 shutdown_connections 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd3410a8c0 0x7fbd341a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbd180a3cc0 0x7fbd180a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd34075420 0x7fbd3419aca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 --2- 172.21.15.19:0/490355800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd340769f0 0x7fbd3419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 >> 172.21.15.19:0/490355800 conn(0x7fbd340fdf80 msgr2=0x7fbd34106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 shutdown_connections 2024-04-25T15:03:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:49.770+0000 7fbd38a00700 1 -- 172.21.15.19:0/490355800 wait complete. 2024-04-25T15:03:50.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:50 smithi019 ceph-mon[28090]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:50.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T15:03:50.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:50 smithi083 ceph-mon[32919]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:50 smithi102 ceph-mon[33005]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:51.518 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:51 smithi019 ceph-mon[28090]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:51.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:51 smithi083 ceph-mon[32919]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:51 smithi102 ceph-mon[33005]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 -- 172.21.15.19:0/1484850789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4340754a0 msgr2=0x7fa434075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 --2- 172.21.15.19:0/1484850789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4340754a0 0x7fa434075920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098f0 tx=0x7fa424009c00 comp rx=0 tx=0).stop 2024-04-25T15:03:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 -- 172.21.15.19:0/1484850789 shutdown_connections 2024-04-25T15:03:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 --2- 172.21.15.19:0/1484850789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa43410f090 0x7fa434111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 --2- 172.21.15.19:0/1484850789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa434076a70 0x7fa434074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 --2- 172.21.15.19:0/1484850789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4340754a0 0x7fa434075920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 -- 172.21.15.19:0/1484850789 >> 172.21.15.19:0/1484850789 conn(0x7fa4340fe010 msgr2=0x7fa434100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:53.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 -- 172.21.15.19:0/1484850789 shutdown_connections 2024-04-25T15:03:53.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.159+0000 7fa43c800700 1 -- 172.21.15.19:0/1484850789 wait complete. 2024-04-25T15:03:53.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.160+0000 7fa43c800700 1 Processor -- start 2024-04-25T15:03:53.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.160+0000 7fa43c800700 1 -- start start 2024-04-25T15:03:53.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 0x7fa43419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:53.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 0x7fa43419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:53.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa434114b90 con 0x7fa43410f090 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa434114a10 con 0x7fa434076a70 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43c800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa434114d10 con 0x7fa4340754a0 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37048/0 (socket says 172.21.15.19:37048) 2024-04-25T15:03:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa43a800700 1 -- 172.21.15.19:0/1073343408 learned_addr learned my addr 172.21.15.19:0/1073343408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa439e00700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 0x7fa43419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.161+0000 7fa439400700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 0x7fa43419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 -- 172.21.15.19:0/1073343408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 msgr2=0x7fa43419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 0x7fa43419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 -- 172.21.15.19:0/1073343408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 msgr2=0x7fa43419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 0x7fa43419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 -- 172.21.15.19:0/1073343408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4341a6f20 con 0x7fa43410f090 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa439e00700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 0x7fa43419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa439400700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 0x7fa43419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa43a800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa43000ef60 tx=0x7fa43000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.162+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300092e0 con 0x7fa43410f090 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.163+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa430004bb0 con 0x7fa43410f090 2024-04-25T15:03:53.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.163+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4341a71b0 con 0x7fa43410f090 2024-04-25T15:03:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.163+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430004020 con 0x7fa43410f090 2024-04-25T15:03:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.163+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4341a76a0 con 0x7fa43410f090 2024-04-25T15:03:53.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.164+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa434069730 con 0x7fa43410f090 2024-04-25T15:03:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.166+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa43001b030 con 0x7fa43410f090 2024-04-25T15:03:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.167+0000 7fa42ea00700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 0x7fa41c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.168+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fa4300cfe70 con 0x7fa43410f090 2024-04-25T15:03:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.168+0000 7fa439e00700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 0x7fa41c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:53.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.168+0000 7fa439e00700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 0x7fa41c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa428009750 tx=0x7fa428009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.172+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa430022020 con 0x7fa43410f090 2024-04-25T15:03:53.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.411+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa434062f40 con 0x7fa41c0a3c40 2024-04-25T15:03:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.427+0000 7fa42ea00700 1 -- 172.21.15.19:0/1073343408 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fa434062f40 con 0x7fa41c0a3c40 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (2m) 24s ago 2m 17.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6383a965d873 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi083 running (2m) 51s ago 119s 25.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 cbb6e2db6811 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi102 running (2m) 51s ago 2m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f1f64be28243 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (2m) 24s ago 2m 12.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 da177b9db64f 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi083 running (2m) 51s ago 2m 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 36772f42a56f 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi102 running (2m) 51s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0c032dc04d87 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (2m) 24s ago 2m 12.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 01c29c231b62 2024-04-25T15:03:53.429 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi083 running (2m) 51s ago 2m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d16373d2bcf0 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi102 running (2m) 51s ago 2m 15.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 170fde5d3b45 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 24s ago 2m 12.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 edce94a2015b 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi083 running (2m) 51s ago 2m 16.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1b6a5c83e209 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi102 running (2m) 51s ago 2m 13.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7d61b42d1a59 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (10m) 24s ago 10m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3efbb3d4c44a 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi083 *:8443,8765 running (8m) 51s ago 8m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 db06a94539c7 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi102 *:8443,8765 running (8m) 51s ago 8m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9c0a4a8f88dc 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (11m) 24s ago 11m 48.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 6526b989094c 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi083 running (8m) 51s ago 8m 41.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 8e289efd1121 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi102 running (9m) 51s ago 9m 41.6M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 6766046cd009 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 24s ago 8m 55.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 716c3036df49 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi083 running (7m) 51s ago 7m 60.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c43909c297b1 2024-04-25T15:03:53.430 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi102 running (7m) 51s ago 7m 49.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8c3a72a43650 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (6m) 24s ago 6m 45.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 7dac251e3c8d 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi083 running (6m) 51s ago 6m 53.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c754c31279ba 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi102 running (5m) 51s ago 5m 46.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6c66127c3067 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (5m) 24s ago 5m 49.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 bb54c6496b4a 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi083 running (4m) 51s ago 4m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 194adac4c321 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi102 running (4m) 51s ago 4m 58.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ff652537cd3b 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 24s ago 3m 42.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b5e9fcdf62a8 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi083 running (3m) 51s ago 3m 46.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a48d9febf899 2024-04-25T15:03:53.431 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi102 running (2m) 51s ago 2m 51.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 208ee67e6ea2 2024-04-25T15:03:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 msgr2=0x7fa41c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 0x7fa41c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa428009750 tx=0x7fa428009310 comp rx=0 tx=0).stop 2024-04-25T15:03:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 msgr2=0x7fa4341a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa43000ef60 tx=0x7fa43000ef90 comp rx=0 tx=0).stop 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 shutdown_connections 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fa41c0a3c40 0x7fa41c0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa43410f090 0x7fa4341a6880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4340754a0 0x7fa43419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 --2- 172.21.15.19:0/1073343408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa434076a70 0x7fa43419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 >> 172.21.15.19:0/1073343408 conn(0x7fa4340fe010 msgr2=0x7fa434102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 shutdown_connections 2024-04-25T15:03:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:53.431+0000 7fa43c800700 1 -- 172.21.15.19:0/1073343408 wait complete. 2024-04-25T15:03:53.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:53 smithi019 ceph-mon[28090]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:53.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:53 smithi083 ceph-mon[32919]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:53.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:53 smithi102 ceph-mon[33005]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:54.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T15:03:54.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:54 smithi019 ceph-mon[28090]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:54.778 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:54.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:54 smithi083 ceph-mon[32919]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:54 smithi102 ceph-mon[33005]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:55.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:55 smithi019 ceph-mon[28090]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:55.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:55 smithi083 ceph-mon[32919]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:55.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:55 smithi102 ceph-mon[33005]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 -- 172.21.15.19:0/2168037276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c103cf0 msgr2=0x7fda7c104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 --2- 172.21.15.19:0/2168037276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c103cf0 0x7fda7c104150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fda680099c0 tx=0x7fda68009cd0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 -- 172.21.15.19:0/2168037276 shutdown_connections 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 --2- 172.21.15.19:0/2168037276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c104690 0x7fda7c10af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 --2- 172.21.15.19:0/2168037276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c102af0 0x7fda7c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 --2- 172.21.15.19:0/2168037276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c103cf0 0x7fda7c104150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.118+0000 7fda84400700 1 -- 172.21.15.19:0/2168037276 >> 172.21.15.19:0/2168037276 conn(0x7fda7c0fe090 msgr2=0x7fda7c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.119+0000 7fda84400700 1 -- 172.21.15.19:0/2168037276 shutdown_connections 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.119+0000 7fda84400700 1 -- 172.21.15.19:0/2168037276 wait complete. 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.120+0000 7fda84400700 1 Processor -- start 2024-04-25T15:03:57.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.120+0000 7fda84400700 1 -- start start 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c102af0 0x7fda7c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c103cf0 0x7fda7c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10e5e0 con 0x7fda7c103cf0 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10e460 con 0x7fda7c104690 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda84400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c10e760 con 0x7fda7c102af0 2024-04-25T15:03:57.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:57.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:43218/0 (socket says 172.21.15.19:43218) 2024-04-25T15:03:57.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 -- 172.21.15.19:0/4094755157 learned_addr learned my addr 172.21.15.19:0/4094755157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:03:57.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda81a00700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c102af0 0x7fda7c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda81000700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c103cf0 0x7fda7c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 -- 172.21.15.19:0/4094755157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c102af0 msgr2=0x7fda7c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c102af0 0x7fda7c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 -- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c103cf0 msgr2=0x7fda7c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.121+0000 7fda82400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c103cf0 0x7fda7c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda82400700 1 -- 172.21.15.19:0/4094755157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c1a2ae0 con 0x7fda7c104690 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda82400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e940 tx=0x7fda7400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74004a30 con 0x7fda7c104690 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda740054d0 con 0x7fda7c104690 2024-04-25T15:03:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c1a2dd0 con 0x7fda7c104690 2024-04-25T15:03:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400cce0 con 0x7fda7c104690 2024-04-25T15:03:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.122+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c1a32d0 con 0x7fda7c104690 2024-04-25T15:03:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.124+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c069730 con 0x7fda7c104690 2024-04-25T15:03:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.125+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda74005020 con 0x7fda7c104690 2024-04-25T15:03:57.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.127+0000 7fda7a000700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 0x7fda640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:03:57.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.127+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fda740d0490 con 0x7fda7c104690 2024-04-25T15:03:57.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.127+0000 7fda81a00700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 0x7fda640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:03:57.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.128+0000 7fda81a00700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 0x7fda640a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda7c103b50 tx=0x7fda7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:03:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.131+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda7409ce20 con 0x7fda7c104690 2024-04-25T15:03:57.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:57 smithi019 ceph-mon[28090]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.377+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fda7c076390 con 0x7fda640a3cc0 2024-04-25T15:03:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.388+0000 7fda7a000700 1 -- 172.21.15.19:0/4094755157 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fda7c076390 con 0x7fda640a3cc0 2024-04-25T15:03:57.389 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T15:03:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 55s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-04-25T15:03:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 55s ago 8m smithi019=x;smithi083=y;smithi102=z;count:3 2024-04-25T15:03:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 55s ago 9m smithi019:172.21.15.19=a;smithi083:172.21.15.83=b;smithi102:172.21.15.102=c;count:3 2024-04-25T15:03:57.390 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 55s ago - 2024-04-25T15:03:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 msgr2=0x7fda640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 0x7fda640a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda7c103b50 tx=0x7fda7000b040 comp rx=0 tx=0).stop 2024-04-25T15:03:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 msgr2=0x7fda7c1a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e940 tx=0x7fda7400ec50 comp rx=0 tx=0).stop 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 shutdown_connections 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda7c104690 0x7fda7c1a2440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda7c102af0 0x7fda7c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fda640a3cc0 0x7fda640a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 --2- 172.21.15.19:0/4094755157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda7c103cf0 0x7fda7c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 >> 172.21.15.19:0/4094755157 conn(0x7fda7c0fe090 msgr2=0x7fda7c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 shutdown_connections 2024-04-25T15:03:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:03:57.391+0000 7fda84400700 1 -- 172.21.15.19:0/4094755157 wait complete. 2024-04-25T15:03:57.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:57 smithi083 ceph-mon[32919]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:57.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:57 smithi102 ceph-mon[33005]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:58.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T15:03:58.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:58 smithi019 ceph-mon[28090]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:03:58.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:58 smithi083 ceph-mon[32919]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:58 smithi102 ceph-mon[33005]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:03:59.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:03:59 smithi019 ceph-mon[28090]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:59.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:03:59 smithi083 ceph-mon[32919]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:59.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:03:59 smithi102 ceph-mon[33005]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:00.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 -- 172.21.15.19:0/1159999500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10104ce0 msgr2=0x7f2a10105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 --2- 172.21.15.19:0/1159999500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10104ce0 0x7f2a10105100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a000099c0 tx=0x7f2a00009cd0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 -- 172.21.15.19:0/1159999500 shutdown_connections 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 --2- 172.21.15.19:0/1159999500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 0x7f2a1010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 --2- 172.21.15.19:0/1159999500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10104ce0 0x7f2a10105100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 --2- 172.21.15.19:0/1159999500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10105ee0 0x7f2a10106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 -- 172.21.15.19:0/1159999500 >> 172.21.15.19:0/1159999500 conn(0x7f2a101002a0 msgr2=0x7f2a101026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 -- 172.21.15.19:0/1159999500 shutdown_connections 2024-04-25T15:04:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.909+0000 7f2a17200700 1 -- 172.21.15.19:0/1159999500 wait complete. 2024-04-25T15:04:00.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.910+0000 7f2a17200700 1 Processor -- start 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 -- start start 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 0x7f2a1019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 0x7f2a101a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10110750 con 0x7f2a10105ee0 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101105d0 con 0x7f2a10104ce0 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a17200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101108d0 con 0x7f2a10106880 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a07e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 0x7f2a1019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:43244/0 (socket says 172.21.15.19:43244) 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.911+0000 7f2a0fe00700 1 -- 172.21.15.19:0/2454852009 learned_addr learned my addr 172.21.15.19:0/2454852009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a15200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 0x7f2a101a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:00.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 -- 172.21.15.19:0/2454852009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 msgr2=0x7f2a101a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 0x7f2a101a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 -- 172.21.15.19:0/2454852009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 msgr2=0x7f2a1019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 -- 172.21.15.19:0/2454852009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a101a2a00 con 0x7f2a10105ee0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a0fe00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a07e00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 0x7f2a1019b2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29f800e9f0 tx=0x7f29f800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.912+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f800c9c0 con 0x7f2a10105ee0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.913+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f800cb60 con 0x7f2a10105ee0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.913+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a101a2cf0 con 0x7f2a10105ee0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.913+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8018680 con 0x7f2a10105ee0 2024-04-25T15:04:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.913+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a101a3180 con 0x7f2a10105ee0 2024-04-25T15:04:00.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.914+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1002df80 con 0x7f2a10105ee0 2024-04-25T15:04:00.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.916+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f29f8010430 con 0x7f2a10105ee0 2024-04-25T15:04:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.917+0000 7f2a0d600700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 0x7f29f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.918+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f29f80d05a0 con 0x7f2a10105ee0 2024-04-25T15:04:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.918+0000 7f2a0fe00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 0x7f29f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.919+0000 7f2a0fe00700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 0x7f29f40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a00000c00 tx=0x7f2a00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:00.923+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29f809cf30 con 0x7f2a10105ee0 2024-04-25T15:04:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.167+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2a10062e80 con 0x7f29f40a3c40 2024-04-25T15:04:01.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.171+0000 7f2a0d600700 1 -- 172.21.15.19:0/2454852009 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2a10062e80 con 0x7f29f40a3c40 2024-04-25T15:04:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-04-25T15:04:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-04-25T15:04:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:smithi083 172.21.15.83 2024-04-25T15:04:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:smithi102 172.21.15.102 2024-04-25T15:04:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-04-25T15:04:01.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.174+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 msgr2=0x7f29f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:01.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.174+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 0x7f29f40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a00000c00 tx=0x7f2a00005fb0 comp rx=0 tx=0).stop 2024-04-25T15:04:01.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 msgr2=0x7f2a1019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:01.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 0x7f2a1019b2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29f800e9f0 tx=0x7f29f800ed00 comp rx=0 tx=0).stop 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 shutdown_connections 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a10106880 0x7f2a101a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a10104ce0 0x7f2a1019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f29f40a3c40 0x7f29f40a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 --2- 172.21.15.19:0/2454852009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a10105ee0 0x7f2a1019b2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 >> 172.21.15.19:0/2454852009 conn(0x7f2a101002a0 msgr2=0x7f2a101091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 shutdown_connections 2024-04-25T15:04:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:01.175+0000 7f2a17200700 1 -- 172.21.15.19:0/2454852009 wait complete. 2024-04-25T15:04:01.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:01 smithi019 ceph-mon[28090]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:01.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:01 smithi083 ceph-mon[32919]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:01.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:01 smithi102 ceph-mon[33005]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:02.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T15:04:02.475 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:02.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:02 smithi019 ceph-mon[28090]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:02.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:02 smithi083 ceph-mon[32919]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:02 smithi102 ceph-mon[33005]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:03.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:03 smithi019 ceph-mon[28090]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:03.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:03 smithi083 ceph-mon[32919]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:03.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:03 smithi102 ceph-mon[33005]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 -- 172.21.15.19:0/2398204962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 msgr2=0x7f220c074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 --2- 172.21.15.19:0/2398204962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c074ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009990 tx=0x7f21fc009ca0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 -- 172.21.15.19:0/2398204962 shutdown_connections 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 --2- 172.21.15.19:0/2398204962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c10efc0 0x7f220c1113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 --2- 172.21.15.19:0/2398204962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c074ed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 --2- 172.21.15.19:0/2398204962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c0754a0 0x7f220c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 -- 172.21.15.19:0/2398204962 >> 172.21.15.19:0/2398204962 conn(0x7f220c0fdf80 msgr2=0x7f220c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 -- 172.21.15.19:0/2398204962 shutdown_connections 2024-04-25T15:04:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.735+0000 7f2211a00700 1 -- 172.21.15.19:0/2398204962 wait complete. 2024-04-25T15:04:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.736+0000 7f2211a00700 1 Processor -- start 2024-04-25T15:04:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 -- start start 2024-04-25T15:04:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:04.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c10efc0 0x7f220c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f220c114ad0 con 0x7f220c076a70 2024-04-25T15:04:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f220c114950 con 0x7f220c0754a0 2024-04-25T15:04:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2211a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f220c114c50 con 0x7f220c10efc0 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f2203e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.737+0000 7f220be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:50328/0 (socket says 172.21.15.19:50328) 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 -- 172.21.15.19:0/2054916027 learned_addr learned my addr 172.21.15.19:0/2054916027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f2211000700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c10efc0 0x7f220c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 -- 172.21.15.19:0/2054916027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c10efc0 msgr2=0x7f220c1a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c10efc0 0x7f220c1a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 -- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 msgr2=0x7f220c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:04.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 -- 172.21.15.19:0/2054916027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c1a6df0 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f2203e00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.738+0000 7f220be00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0047b0 tx=0x7f21fc009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.739+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc026070 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.739+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c1a7080 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.739+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21fc021410 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.739+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc017650 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.739+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a7530 con 0x7f220c0754a0 2024-04-25T15:04:04.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.740+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c069730 con 0x7f220c0754a0 2024-04-25T15:04:04.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.742+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f21fc029070 con 0x7f220c0754a0 2024-04-25T15:04:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.744+0000 7f2209600700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 0x7f21f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.744+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f21fc0d1950 con 0x7f220c0754a0 2024-04-25T15:04:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.744+0000 7f2203e00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 0x7f21f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:04.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.745+0000 7f2203e00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 0x7f21f00a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f220c0767a0 tx=0x7f21f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:04.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.746+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21fc030070 con 0x7f220c0754a0 2024-04-25T15:04:04.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.988+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f220c062e80 con 0x7f21f00a3c40 2024-04-25T15:04:04.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.993+0000 7f2209600700 1 -- 172.21.15.19:0/2054916027 <== mgr.14152 v2:172.21.15.19:6800/983233775 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f220c062e80 con 0x7f21f00a3c40 2024-04-25T15:04:04.994 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T15:04:04.994 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T15:04:04.994 INFO:teuthology.orchestra.run.smithi019.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T15:04:04.995 INFO:teuthology.orchestra.run.smithi019.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T15:04:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 msgr2=0x7f21f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 0x7f21f00a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f220c0767a0 tx=0x7f21f4009260 comp rx=0 tx=0).stop 2024-04-25T15:04:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 msgr2=0x7f220c19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0047b0 tx=0x7f21fc009510 comp rx=0 tx=0).stop 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 shutdown_connections 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f220c10efc0 0x7f220c1a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f220c0754a0 0x7f220c19f190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f21f00a3c40 0x7f21f00a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 --2- 172.21.15.19:0/2054916027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f220c076a70 0x7f220c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 >> 172.21.15.19:0/2054916027 conn(0x7f220c0fdf80 msgr2=0x7f220c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.996+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 shutdown_connections 2024-04-25T15:04:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:04.997+0000 7f2211a00700 1 -- 172.21.15.19:0/2054916027 wait complete. 2024-04-25T15:04:05.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:05 smithi019 ceph-mon[28090]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:05.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:05 smithi083 ceph-mon[32919]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:05.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:05 smithi102 ceph-mon[33005]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:05.817 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:04:05.825 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:05.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T15:04:06.288 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:06 smithi019 ceph-mon[28090]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:06.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:06 smithi083 ceph-mon[32919]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:06 smithi102 ceph-mon[33005]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:07.497 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:07 smithi019 ceph-mon[28090]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:07.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:07 smithi083 ceph-mon[32919]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:07.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:07 smithi102 ceph-mon[33005]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 -- 172.21.15.19:0/979608940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8101320 msgr2=0x7f4df8103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 --2- 172.21.15.19:0/979608940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8101320 0x7f4df8103750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009990 tx=0x7f4de8009ca0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 -- 172.21.15.19:0/979608940 shutdown_connections 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 --2- 172.21.15.19:0/979608940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df8106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 --2- 172.21.15.19:0/979608940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8101320 0x7f4df8103750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 --2- 172.21.15.19:0/979608940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8103c90 0x7f4df8106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 -- 172.21.15.19:0/979608940 >> 172.21.15.19:0/979608940 conn(0x7f4df80fcf40 msgr2=0x7f4df80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 -- 172.21.15.19:0/979608940 shutdown_connections 2024-04-25T15:04:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.589+0000 7f4dfca00700 1 -- 172.21.15.19:0/979608940 wait complete. 2024-04-25T15:04:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.590+0000 7f4dfca00700 1 Processor -- start 2024-04-25T15:04:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.590+0000 7f4dfca00700 1 -- start start 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 0x7f4df819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df81a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8114ac0 con 0x7f4df8103c90 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8114940 con 0x7f4df8106660 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4dfca00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8114c40 con 0x7f4df8101320 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 0x7f4df819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:53528/0 (socket says 172.21.15.19:53528) 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 -- 172.21.15.19:0/858467628 learned_addr learned my addr 172.21.15.19:0/858467628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df6800700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df81a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 -- 172.21.15.19:0/858467628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 msgr2=0x7f4df81a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df81a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 -- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 msgr2=0x7f4df819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 0x7f4df819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5e00700 1 -- 172.21.15.19:0/858467628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81a6e60 con 0x7f4df8101320 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df5400700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 0x7f4df819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.591+0000 7f4df6800700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df81a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4df5e00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8026070 con 0x7f4df8101320 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de8021410 con 0x7f4df8101320 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81a70f0 con 0x7f4df8101320 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017650 con 0x7f4df8101320 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.592+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a7600 con 0x7f4df8101320 2024-04-25T15:04:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.593+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df8069730 con 0x7f4df8101320 2024-04-25T15:04:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.595+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4de8029070 con 0x7f4df8101320 2024-04-25T15:04:08.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.597+0000 7f4deea00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 0x7f4ddc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:08.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.597+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4de80d1570 con 0x7f4df8101320 2024-04-25T15:04:08.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.597+0000 7f4df5400700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 0x7f4ddc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:08.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.598+0000 7f4df5400700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 0x7f4ddc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4de4011490 tx=0x7f4de4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.599+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4de8013070 con 0x7f4df8101320 2024-04-25T15:04:08.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.878+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4df802df80 con 0x7f4df8101320 2024-04-25T15:04:08.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.879+0000 7f4deea00700 1 -- 172.21.15.19:0/858467628 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3006 (secure 0 0 0) 0x7f4de809df90 con 0x7f4df8101320 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:e20 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:04:08.882 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 20 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T15:02:00.518771+0000 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T15:02:04.773178+0000 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-04-25T15:04:08.883 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 1 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:in 0 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24590} 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-04-25T15:04:08.884 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24590 members: 24590 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24590} state up:active seq 3 addr [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{0:24572} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/2613932433,v1:172.21.15.19:6841/2613932433] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.19:6836/3801236532,v1:172.21.15.19:6837/3801236532] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{-1:14451} state up:standby seq 1 addr [v2:172.21.15.19:6838/2415265237,v1:172.21.15.19:6839/2415265237] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.102:6832/1625313784,v1:172.21.15.102:6833/1625313784] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{-1:24455} state up:standby seq 1 addr [v2:172.21.15.83:6832/2427216282,v1:172.21.15.83:6833/2427216282] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:24473} state up:standby seq 1 addr [v2:172.21.15.19:6834/1454655799,v1:172.21.15.19:6835/1454655799] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{-1:24503} state up:standby seq 1 addr [v2:172.21.15.83:6834/334283941,v1:172.21.15.83:6835/334283941] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.885 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{-1:24514} state up:standby seq 1 addr [v2:172.21.15.102:6834/3800814259,v1:172.21.15.102:6835/3800814259] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.886 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{-1:24526} state up:standby seq 1 addr [v2:172.21.15.102:6836/560569157,v1:172.21.15.102:6837/560569157] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.886 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{-1:24533} state up:standby seq 1 addr [v2:172.21.15.83:6836/3676268156,v1:172.21.15.83:6837/3676268156] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.886 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{-1:24550} state up:standby seq 1 addr [v2:172.21.15.102:6838/4066270851,v1:172.21.15.102:6839/4066270851] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:04:08.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 msgr2=0x7f4ddc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 0x7f4ddc0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4de4011490 tx=0x7f4de4009260 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 msgr2=0x7f4df819f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8009580 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 shutdown_connections 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df8106660 0x7f4df81a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4df8101320 0x7f4df819f250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4ddc0a3c40 0x7f4ddc0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 --2- 172.21.15.19:0/858467628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4df8103c90 0x7f4df819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.882+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 >> 172.21.15.19:0/858467628 conn(0x7f4df80fcf40 msgr2=0x7f4df8104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.883+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 shutdown_connections 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:08.883+0000 7f4dfca00700 1 -- 172.21.15.19:0/858467628 wait complete. 2024-04-25T15:04:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:04:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:09 smithi019 ceph-mon[28090]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:09.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:09 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/858467628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:04:09.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T15:04:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:09 smithi083 ceph-mon[32919]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:09 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/858467628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:04:09.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:09 smithi102 ceph-mon[33005]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:09.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:09 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/858467628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:04:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:11.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:11 smithi019 ceph-mon[28090]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:11.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:11 smithi083 ceph-mon[32919]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:11.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:11 smithi102 ceph-mon[33005]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:12.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 -- 172.21.15.19:0/862658697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 msgr2=0x7fe460109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 --2- 172.21.15.19:0/862658697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe460109500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe450009920 tx=0x7fe450009c30 comp rx=0 tx=0).stop 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 -- 172.21.15.19:0/862658697 shutdown_connections 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 --2- 172.21.15.19:0/862658697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 0x7fe460111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 --2- 172.21.15.19:0/862658697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe460109500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 --2- 172.21.15.19:0/862658697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 0x7fe46010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 -- 172.21.15.19:0/862658697 >> 172.21.15.19:0/862658697 conn(0x7fe4600780b0 msgr2=0x7fe46007a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 -- 172.21.15.19:0/862658697 shutdown_connections 2024-04-25T15:04:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.216+0000 7fe466c00700 1 -- 172.21.15.19:0/862658697 wait complete. 2024-04-25T15:04:12.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 Processor -- start 2024-04-25T15:04:12.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 -- start start 2024-04-25T15:04:12.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:12.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 0x7fe46019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:12.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 0x7fe4601a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:12.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe460114b60 con 0x7fe46010ac80 2024-04-25T15:04:12.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4601149e0 con 0x7fe4601090e0 2024-04-25T15:04:12.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.218+0000 7fe466c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe460114ce0 con 0x7fe46010a2e0 2024-04-25T15:04:12.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe464c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 0x7fe4601a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45f400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 0x7fe46019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:50344/0 (socket says 172.21.15.19:50344) 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 -- 172.21.15.19:0/797855094 learned_addr learned my addr 172.21.15.19:0/797855094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 -- 172.21.15.19:0/797855094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 msgr2=0x7fe46019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 0x7fe46019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 -- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 msgr2=0x7fe4601a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 0x7fe4601a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.219+0000 7fe45fe00700 1 -- 172.21.15.19:0/797855094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4601a6ee0 con 0x7fe4601090e0 2024-04-25T15:04:12.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe45fe00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe450009230 tx=0x7fe450005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450027070 con 0x7fe4601090e0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe450005510 con 0x7fe4601090e0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4500049c0 con 0x7fe4601090e0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4601a7170 con 0x7fe4601090e0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.220+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a7570 con 0x7fe4601090e0 2024-04-25T15:04:12.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.221+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe460069730 con 0x7fe4601090e0 2024-04-25T15:04:12.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.224+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe45002a070 con 0x7fe4601090e0 2024-04-25T15:04:12.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.225+0000 7fe45cc00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 0x7fe4440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:12.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.225+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe4500d1200 con 0x7fe4601090e0 2024-04-25T15:04:12.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.225+0000 7fe45f400700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 0x7fe4440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:12.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.226+0000 7fe45f400700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 0x7fe4440a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005fd0 tx=0x7fe44c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:12.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.227+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe450031020 con 0x7fe4601090e0 2024-04-25T15:04:12.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.461+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe460063650 con 0x7fe4601090e0 2024-04-25T15:04:12.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.462+0000 7fe45cc00700 1 -- 172.21.15.19:0/797855094 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5259 (secure 0 0 0) 0x7fe450099030 con 0x7fe4601090e0 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 90 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T14:52:38.625810+0000 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T15:02:57.459165+0000 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-04-25T15:04:12.465 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/1027909993,v1:172.21.15.19:6803/1027909993] [v2:172.21.15.19:6804/1027909993,v1:172.21.15.19:6805/1027909993] exists,up 8649bfaf-7456-4969-a41c-c8752a4bf853 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/3419816052,v1:172.21.15.83:6801/3419816052] [v2:172.21.15.83:6802/3419816052,v1:172.21.15.83:6803/3419816052] exists,up fb5271f6-eea9-4b6f-8b71-6c3f259b237e 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/379851586,v1:172.21.15.102:6801/379851586] [v2:172.21.15.102:6802/379851586,v1:172.21.15.102:6803/379851586] exists,up 3e2326e2-31a4-4563-9b95-e5da5099e1f9 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/3094191876,v1:172.21.15.19:6811/3094191876] [v2:172.21.15.19:6812/3094191876,v1:172.21.15.19:6813/3094191876] exists,up 05431369-edfd-411e-9a63-b6b60b86ae1d 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/683494368,v1:172.21.15.83:6809/683494368] [v2:172.21.15.83:6810/683494368,v1:172.21.15.83:6811/683494368] exists,up 45e8783f-267c-42c5-834c-905d5e715a05 2024-04-25T15:04:12.466 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/4087768,v1:172.21.15.102:6809/4087768] [v2:172.21.15.102:6810/4087768,v1:172.21.15.102:6811/4087768] exists,up 23ba29b0-bb4a-458a-b988-de82bd6f7629 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/3457660451,v1:172.21.15.19:6819/3457660451] [v2:172.21.15.19:6820/3457660451,v1:172.21.15.19:6821/3457660451] exists,up e7e6a2a9-70b8-48a9-8dff-2b6f2d164734 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/1101145206,v1:172.21.15.83:6817/1101145206] [v2:172.21.15.83:6818/1101145206,v1:172.21.15.83:6819/1101145206] exists,up 2b6e2fed-875b-495d-aeae-bd3d93a14fa2 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 62 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/3832909969,v1:172.21.15.102:6817/3832909969] [v2:172.21.15.102:6818/3832909969,v1:172.21.15.102:6819/3832909969] exists,up 3b498ef1-2388-4402-9170-5c4a8e4cc29b 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 70 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/2037522312,v1:172.21.15.19:6827/2037522312] [v2:172.21.15.19:6828/2037522312,v1:172.21.15.19:6829/2037522312] exists,up 7772cbd2-d2b5-4c1b-beaf-93651cd80d09 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 75 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6824/2320025871,v1:172.21.15.83:6825/2320025871] [v2:172.21.15.83:6826/2320025871,v1:172.21.15.83:6827/2320025871] exists,up 924537e9-24c0-45be-8ae2-f5623c73902b 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/2593361750,v1:172.21.15.102:6825/2593361750] [v2:172.21.15.102:6826/2593361750,v1:172.21.15.102:6827/2593361750] exists,up 6578f249-8ff7-4cd6-b80f-9b7b873d0bc2 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.c [2,1] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.f [8,11] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1d [2,4] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1d [4,8] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.25 [5,9] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T15:04:12.467 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3140116712 expires 2024-04-26T14:53:55.852566+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4125323184 expires 2024-04-26T14:53:55.852566+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2740045356 expires 2024-04-26T14:53:55.852566+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/645310036 expires 2024-04-26T14:53:55.852566+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3512004662 expires 2024-04-26T14:53:20.229663+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1703501063 expires 2024-04-26T14:53:20.229663+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2740045356 expires 2024-04-26T14:53:55.852566+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2453471633 expires 2024-04-26T14:53:20.229663+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2453471633 expires 2024-04-26T14:53:20.229663+0000 2024-04-25T15:04:12.468 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/2483228355 expires 2024-04-26T14:53:20.229663+0000 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 msgr2=0x7fe4440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 0x7fe4440a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005fd0 tx=0x7fe44c009310 comp rx=0 tx=0).stop 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 msgr2=0x7fe46019f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe450009230 tx=0x7fe450005270 comp rx=0 tx=0).stop 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 shutdown_connections 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fe4440a3cc0 0x7fe4440a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe46010ac80 0x7fe4601a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4601090e0 0x7fe46019f280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 --2- 172.21.15.19:0/797855094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46010a2e0 0x7fe46019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 >> 172.21.15.19:0/797855094 conn(0x7fe4600780b0 msgr2=0x7fe46010d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.465+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 shutdown_connections 2024-04-25T15:04:12.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:12.466+0000 7fe466c00700 1 -- 172.21.15.19:0/797855094 wait complete. 2024-04-25T15:04:13.279 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T15:04:13.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.516+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2856557735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc1051e0 msgr2=0x7fcdbc105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.516+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2856557735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc105660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009900 tx=0x7fcda802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2856557735 shutdown_connections 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2856557735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2856557735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc105660 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2856557735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2856557735 >> 172.21.15.19:0/2856557735 conn(0x7fcdbc0ff560 msgr2=0x7fcdbc1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2856557735 shutdown_connections 2024-04-25T15:04:13.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.517+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2856557735 wait complete. 2024-04-25T15:04:13.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 Processor -- start 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 -- start start 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc10fb80 con 0x7fcdbc103ff0 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc10fa00 con 0x7fcdbc105ba0 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc2a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc10fd00 con 0x7fcdbc1051e0 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.518+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:50372/0 (socket says 172.21.15.19:50372) 2024-04-25T15:04:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39090/0 (socket says 172.21.15.19:39090) 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 -- 172.21.15.19:0/2458685884 learned_addr learned my addr 172.21.15.19:0/2458685884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc0c00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 -- 172.21.15.19:0/2458685884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc1051e0 msgr2=0x7fcdbc19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 -- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 msgr2=0x7fcdbc19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 -- 172.21.15.19:0/2458685884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc1a1e50 con 0x7fcdbc105ba0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc1600640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdc2000640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400eb10 tx=0x7fcdb400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.519+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4004b20 con 0x7fcdbc105ba0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.520+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb4004cc0 con 0x7fcdbc105ba0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.520+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc1a2140 con 0x7fcdbc105ba0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.520+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4010670 con 0x7fcdbc105ba0 2024-04-25T15:04:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.520+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a2660 con 0x7fcdbc105ba0 2024-04-25T15:04:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.521+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc10b390 con 0x7fcdbc105ba0 2024-04-25T15:04:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.523+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcdb4005020 con 0x7fcdbc105ba0 2024-04-25T15:04:13.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.524+0000 7fcdba000640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 0x7fcda40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.524+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fcdb40d0980 con 0x7fcdbc105ba0 2024-04-25T15:04:13.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.524+0000 7fcdc1600640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 0x7fcda40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.525+0000 7fcdc1600640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 0x7fcda40a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc105040 tx=0x7fcdb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:13.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.529+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdb409d310 con 0x7fcdbc105ba0 2024-04-25T15:04:13.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:13 smithi019 ceph-mon[28090]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:13.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:13 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/797855094' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:04:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.736+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdbc062b50 con 0x7fcdbc105ba0 2024-04-25T15:04:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.739+0000 7fcdba000640 1 -- 172.21.15.19:0/2458685884 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7fcdb409cc10 con 0x7fcdbc105ba0 2024-04-25T15:04:13.741 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:13.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 msgr2=0x7fcda40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 0x7fcda40a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc105040 tx=0x7fcdb0005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 msgr2=0x7fcdbc1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400eb10 tx=0x7fcdb400c2d0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 shutdown_connections 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fcda40a3c30 0x7fcda40a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdbc105ba0 0x7fcdbc1a17e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc1051e0 0x7fcdbc19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 --2- 172.21.15.19:0/2458685884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdbc103ff0 0x7fcdbc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.741+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 >> 172.21.15.19:0/2458685884 conn(0x7fcdbc0ff560 msgr2=0x7fcdbc108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.742+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 shutdown_connections 2024-04-25T15:04:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.742+0000 7fcdc2a00640 1 -- 172.21.15.19:0/2458685884 wait complete. 2024-04-25T15:04:13.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:04:13.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:04:13.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:13 smithi083 ceph-mon[32919]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:13.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:13 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/797855094' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:04:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:13 smithi102 ceph-mon[33005]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:13.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:13 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/797855094' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:04:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 -- 172.21.15.19:0/2259324399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000107b50 msgr2=0x7ff000109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 --2- 172.21.15.19:0/2259324399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000107b50 0x7ff000109fa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feff8009ca0 tx=0x7feff802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 -- 172.21.15.19:0/2259324399 shutdown_connections 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 --2- 172.21.15.19:0/2259324399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000107b50 0x7ff000109fa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 --2- 172.21.15.19:0/2259324399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000105220 0x7ff000107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 --2- 172.21.15.19:0/2259324399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 -- 172.21.15.19:0/2259324399 >> 172.21.15.19:0/2259324399 conn(0x7ff0000fc300 msgr2=0x7ff0000fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 -- 172.21.15.19:0/2259324399 shutdown_connections 2024-04-25T15:04:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.982+0000 7ff006600640 1 -- 172.21.15.19:0/2259324399 wait complete. 2024-04-25T15:04:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 Processor -- start 2024-04-25T15:04:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 -- start start 2024-04-25T15:04:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 0x7ff000198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000107b50 0x7ff00019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010da10 con 0x7ff0001028f0 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010d890 con 0x7ff000107b50 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.983+0000 7ff006600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010db90 con 0x7ff000105220 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49042/0 (socket says 172.21.15.19:49042) 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 -- 172.21.15.19:0/2491634128 learned_addr learned my addr 172.21.15.19:0/2491634128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7feff7e00640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 0x7ff000198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005c00640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000107b50 0x7ff00019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 -- 172.21.15.19:0/2491634128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 msgr2=0x7ff000198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 0x7ff000198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 -- 172.21.15.19:0/2491634128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000107b50 msgr2=0x7ff00019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:13.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000107b50 0x7ff00019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7ff005200640 1 -- 172.21.15.19:0/2491634128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00019fc70 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.984+0000 7feff7e00640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 0x7ff000198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7ff005200640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feff000d950 tx=0x7feff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0017070 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff0009c30 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000f620 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00019ff60 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.985+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a03c0 con 0x7ff0001028f0 2024-04-25T15:04:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.986+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff000069110 con 0x7ff0001028f0 2024-04-25T15:04:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.988+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feff0010460 con 0x7ff0001028f0 2024-04-25T15:04:13.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.989+0000 7feffe000640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:13.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.990+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7feff00cffd0 con 0x7ff0001028f0 2024-04-25T15:04:13.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.990+0000 7feff7e00640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:13.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.990+0000 7feff7e00640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 0x7fefe40a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff000199560 tx=0x7fefe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:13.994+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff000069110 con 0x7ff0001028f0 2024-04-25T15:04:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.200+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff000062bf0 con 0x7ff0001028f0 2024-04-25T15:04:14.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.201+0000 7feffe000640 1 -- 172.21.15.19:0/2491634128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7feff0009de0 con 0x7ff0001028f0 2024-04-25T15:04:14.203 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:04:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 msgr2=0x7fefe40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 0x7fefe40a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff000199560 tx=0x7fefe8005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 msgr2=0x7ff000198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feff000d950 tx=0x7feff000de10 comp rx=0 tx=0).stop 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 shutdown_connections 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff000107b50 0x7ff00019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff000105220 0x7ff000198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 --2- 172.21.15.19:0/2491634128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0001028f0 0x7ff000198050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 >> 172.21.15.19:0/2491634128 conn(0x7ff0000fc300 msgr2=0x7ff000103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 shutdown_connections 2024-04-25T15:04:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.204+0000 7ff006600640 1 -- 172.21.15.19:0/2491634128 wait complete. 2024-04-25T15:04:14.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:04:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 -- 172.21.15.19:0/3099216972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b01041b0 msgr2=0x7f14b01045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 --2- 172.21.15.19:0/3099216972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b01041b0 0x7f14b01045d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009be0 tx=0x7f14a802a6d0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 -- 172.21.15.19:0/3099216972 shutdown_connections 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 --2- 172.21.15.19:0/3099216972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b0105d60 0x7f14b010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 --2- 172.21.15.19:0/3099216972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b0105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 --2- 172.21.15.19:0/3099216972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b01041b0 0x7f14b01045d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 -- 172.21.15.19:0/3099216972 >> 172.21.15.19:0/3099216972 conn(0x7f14b00ff700 msgr2=0x7f14b0101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 -- 172.21.15.19:0/3099216972 shutdown_connections 2024-04-25T15:04:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.439+0000 7f14b8800640 1 -- 172.21.15.19:0/3099216972 wait complete. 2024-04-25T15:04:14.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 Processor -- start 2024-04-25T15:04:14.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 -- start start 2024-04-25T15:04:14.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 0x7f14b01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010fcf0 con 0x7f14b01053a0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010fb70 con 0x7f14b0105d60 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.440+0000 7f14b8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010fe70 con 0x7f14b01041b0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44190/0 (socket says 172.21.15.19:44190) 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 -- 172.21.15.19:0/266979575 learned_addr learned my addr 172.21.15.19:0/266979575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 0x7f14b01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 -- 172.21.15.19:0/266979575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 msgr2=0x7f14b01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 0x7f14b01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 -- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 msgr2=0x7f14b019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5e00640 1 -- 172.21.15.19:0/266979575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b01a2000 con 0x7f14b01041b0 2024-04-25T15:04:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b5400640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.441+0000 7f14b6800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 0x7f14b01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14b5e00640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14a802acc0 tx=0x7f14a8030de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8041020 con 0x7f14b01041b0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a80318f0 con 0x7f14b01041b0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8036040 con 0x7f14b01041b0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b01a2290 con 0x7f14b01041b0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.442+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01a2730 con 0x7f14b01041b0 2024-04-25T15:04:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.443+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b010b5a0 con 0x7f14b01041b0 2024-04-25T15:04:14.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.445+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f14a8044020 con 0x7f14b01041b0 2024-04-25T15:04:14.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.446+0000 7f14a6a00640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 0x7f149c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.446+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f14a80f11d0 con 0x7f14b01041b0 2024-04-25T15:04:14.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.447+0000 7f14b5400640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 0x7f149c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.447+0000 7f14b5400640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 0x7f149c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14b019b8f0 tx=0x7f14a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.451+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14a804b020 con 0x7f14b01041b0 2024-04-25T15:04:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:14 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2458685884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:14.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:14 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2491634128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:14.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.658+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14b0062b80 con 0x7f14b01041b0 2024-04-25T15:04:14.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.659+0000 7f14a6a00640 1 -- 172.21.15.19:0/266979575 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f14a80bdb60 con 0x7f14b01041b0 2024-04-25T15:04:14.661 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 msgr2=0x7f149c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 0x7f149c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14b019b8f0 tx=0x7f14a0009260 comp rx=0 tx=0).stop 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 msgr2=0x7f14b019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14a802acc0 tx=0x7f14a8030de0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 shutdown_connections 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f149c0a3bb0 0x7f149c0a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b0105d60 0x7f14b01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b01053a0 0x7f14b019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 --2- 172.21.15.19:0/266979575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14b01041b0 0x7f14b019a3e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 >> 172.21.15.19:0/266979575 conn(0x7f14b00ff700 msgr2=0x7f14b0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.661+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 shutdown_connections 2024-04-25T15:04:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.662+0000 7f14b8800640 1 -- 172.21.15.19:0/266979575 wait complete. 2024-04-25T15:04:14.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:04:14.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:14 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2458685884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:14.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:14 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2491634128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:14 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2458685884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:14 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2491634128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 -- 172.21.15.19:0/161362902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0105c40 msgr2=0x7fbcb010c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/161362902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0105c40 0x7fbcb010c4d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009ca0 tx=0x7fbca802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 -- 172.21.15.19:0/161362902 shutdown_connections 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/161362902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0105c40 0x7fbcb010c4d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/161362902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb0105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/161362902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0104090 0x7fbcb01044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 -- 172.21.15.19:0/161362902 >> 172.21.15.19:0/161362902 conn(0x7fbcb00ff600 msgr2=0x7fbcb0101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 -- 172.21.15.19:0/161362902 shutdown_connections 2024-04-25T15:04:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.899+0000 7fbcb6a00640 1 -- 172.21.15.19:0/161362902 wait complete. 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 Processor -- start 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 -- start start 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0105c40 0x7fbcb01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010fbe0 con 0x7fbcb0104090 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010fa60 con 0x7fbcb0105280 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.900+0000 7fbcb6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010fd60 con 0x7fbcb0105c40 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49076/0 (socket says 172.21.15.19:49076) 2024-04-25T15:04:14.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 -- 172.21.15.19:0/4246780055 learned_addr learned my addr 172.21.15.19:0/4246780055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb4c00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb6000640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0105c40 0x7fbcb01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 -- 172.21.15.19:0/4246780055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0105c40 msgr2=0x7fbcb01a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0105c40 0x7fbcb01a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 -- 172.21.15.19:0/4246780055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 msgr2=0x7fbcb019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 -- 172.21.15.19:0/4246780055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb01a1e40 con 0x7fbcb0104090 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb4c00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.901+0000 7fbcb5600640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e9c0 tx=0x7fbca400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.902+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400cd80 con 0x7fbcb0104090 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.902+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca40035f0 con 0x7fbcb0104090 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.902+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4010690 con 0x7fbcb0104090 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.902+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb01a2130 con 0x7fbcb0104090 2024-04-25T15:04:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.902+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a2630 con 0x7fbcb0104090 2024-04-25T15:04:14.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.903+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb010b3b0 con 0x7fbcb0104090 2024-04-25T15:04:14.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.905+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbca40037a0 con 0x7fbcb0104090 2024-04-25T15:04:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.906+0000 7fbcae000640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 0x7fbc980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.907+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbca40d07e0 con 0x7fbcb0104090 2024-04-25T15:04:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.907+0000 7fbcb4c00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 0x7fbc980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.907+0000 7fbcb4c00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 0x7fbc980a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcb019b730 tx=0x7fbca0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:14.911+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbca409d170 con 0x7fbcb0104090 2024-04-25T15:04:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.122+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbcb0062b50 con 0x7fbcb0104090 2024-04-25T15:04:15.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.123+0000 7fbcae000640 1 -- 172.21.15.19:0/4246780055 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fbca409ca70 con 0x7fbcb0104090 2024-04-25T15:04:15.125 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 msgr2=0x7fbc980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 0x7fbc980a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcb019b730 tx=0x7fbca0005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 msgr2=0x7fbcb019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e9c0 tx=0x7fbca400ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 shutdown_connections 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcb0105c40 0x7fbcb01a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fbc980a3c30 0x7fbc980a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb0105280 0x7fbcb019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 --2- 172.21.15.19:0/4246780055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcb0104090 0x7fbcb019a220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.125+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 >> 172.21.15.19:0/4246780055 conn(0x7fbcb00ff600 msgr2=0x7fbcb0108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.126+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 shutdown_connections 2024-04-25T15:04:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.126+0000 7fbcb6a00640 1 -- 172.21.15.19:0/4246780055 wait complete. 2024-04-25T15:04:15.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 -- 172.21.15.19:0/246351096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 msgr2=0x7fc860105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 --2- 172.21.15.19:0/246351096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc860105660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc848009900 tx=0x7fc84802a410 comp rx=0 tx=0).stop 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 -- 172.21.15.19:0/246351096 shutdown_connections 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 --2- 172.21.15.19:0/246351096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 0x7fc86010c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 --2- 172.21.15.19:0/246351096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc860105660 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 --2- 172.21.15.19:0/246351096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc860104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 -- 172.21.15.19:0/246351096 >> 172.21.15.19:0/246351096 conn(0x7fc8600ff560 msgr2=0x7fc8601019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 -- 172.21.15.19:0/246351096 shutdown_connections 2024-04-25T15:04:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.360+0000 7fc864a00640 1 -- 172.21.15.19:0/246351096 wait complete. 2024-04-25T15:04:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.361+0000 7fc864a00640 1 Processor -- start 2024-04-25T15:04:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.361+0000 7fc864a00640 1 -- start start 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc86019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 0x7fc8601a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc86010fb80 con 0x7fc8601051e0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc86010fa00 con 0x7fc860105ba0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc864a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc86010fd00 con 0x7fc860103ff0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc857e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc86019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc857e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc86019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49092/0 (socket says 172.21.15.19:49092) 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44234/0 (socket says 172.21.15.19:44234) 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc857e00640 1 -- 172.21.15.19:0/4245451281 learned_addr learned my addr 172.21.15.19:0/4245451281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85f200640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 0x7fc8601a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 -- 172.21.15.19:0/4245451281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 msgr2=0x7fc8601a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 0x7fc8601a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 -- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 msgr2=0x7fc86019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc86019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.362+0000 7fc85e800640 1 -- 172.21.15.19:0/4245451281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8601a1e50 con 0x7fc860103ff0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc85e800640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9c0 tx=0x7fc85000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000ccc0 con 0x7fc860103ff0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8500035f0 con 0x7fc860103ff0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000ce60 con 0x7fc860103ff0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8601a2140 con 0x7fc860103ff0 2024-04-25T15:04:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.363+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a2640 con 0x7fc860103ff0 2024-04-25T15:04:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.364+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86010b380 con 0x7fc860103ff0 2024-04-25T15:04:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.366+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc850004000 con 0x7fc860103ff0 2024-04-25T15:04:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.368+0000 7fc857400640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.368+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fc8500d07c0 con 0x7fc860103ff0 2024-04-25T15:04:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.368+0000 7fc857e00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.369+0000 7fc857e00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 0x7fc8440a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc86019b740 tx=0x7fc848005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:15.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.372+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc85009d150 con 0x7fc860103ff0 2024-04-25T15:04:15.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.622+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc86002d5c0 con 0x7fc860103ff0 2024-04-25T15:04:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.624+0000 7fc857400640 1 -- 172.21.15.19:0/4245451281 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7fc85009ca50 con 0x7fc860103ff0 2024-04-25T15:04:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:15.626 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:04:15.626 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:04:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 msgr2=0x7fc8440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 0x7fc8440a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc86019b740 tx=0x7fc848005a20 comp rx=0 tx=0).stop 2024-04-25T15:04:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 msgr2=0x7fc86019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9c0 tx=0x7fc85000ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 shutdown_connections 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc860105ba0 0x7fc8601a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8601051e0 0x7fc86019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 --2- 172.21.15.19:0/4245451281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc860103ff0 0x7fc86019a230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 >> 172.21.15.19:0/4245451281 conn(0x7fc8600ff560 msgr2=0x7fc860108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 shutdown_connections 2024-04-25T15:04:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.626+0000 7fc864a00640 1 -- 172.21.15.19:0/4245451281 wait complete. 2024-04-25T15:04:15.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:15.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:15 smithi019 ceph-mon[28090]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:15.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/266979575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4246780055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:15 smithi083 ceph-mon[32919]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/266979575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4246780055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:15 smithi102 ceph-mon[33005]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/266979575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4246780055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.847+0000 7f1533e00640 1 -- 172.21.15.19:0/1893994285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 msgr2=0x7f1534104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.847+0000 7f1533e00640 1 --2- 172.21.15.19:0/1893994285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534104c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1524009900 tx=0x7f152402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 -- 172.21.15.19:0/1893994285 shutdown_connections 2024-04-25T15:04:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 --2- 172.21.15.19:0/1893994285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 0x7f1534109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 --2- 172.21.15.19:0/1893994285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 --2- 172.21.15.19:0/1893994285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534104c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 -- 172.21.15.19:0/1893994285 >> 172.21.15.19:0/1893994285 conn(0x7f15340fc220 msgr2=0x7f15340fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 -- 172.21.15.19:0/1893994285 shutdown_connections 2024-04-25T15:04:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.848+0000 7f1533e00640 1 -- 172.21.15.19:0/1893994285 wait complete. 2024-04-25T15:04:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.849+0000 7f1533e00640 1 Processor -- start 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.849+0000 7f1533e00640 1 -- start start 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 0x7f153419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f153410d920 con 0x7f1534102850 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f153410d7a0 con 0x7f1534105180 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f153410daa0 con 0x7f1534107ab0 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1532a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f152a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1532a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49104/0 (socket says 172.21.15.19:49104) 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1532a00640 1 -- 172.21.15.19:0/1425369371 learned_addr learned my addr 172.21.15.19:0/1425369371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.850+0000 7f1533400640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 0x7f153419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 -- 172.21.15.19:0/1425369371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 msgr2=0x7f153419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 0x7f153419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 -- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 msgr2=0x7f1534198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 -- 172.21.15.19:0/1425369371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153419fc40 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152a000640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534198560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f151c00e9c0 tx=0x7f151c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c00ccc0 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c0108e0 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0187c0 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153419ff30 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.851+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15341a03f0 con 0x7f1534105180 2024-04-25T15:04:15.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.853+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15340fdd10 con 0x7f1534105180 2024-04-25T15:04:15.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.855+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f151c010430 con 0x7f1534105180 2024-04-25T15:04:15.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.856+0000 7f152be00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 0x7f15180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:15.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.856+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f151c0d0710 con 0x7f1534105180 2024-04-25T15:04:15.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.856+0000 7f1532a00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 0x7f15180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:15.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.857+0000 7f1532a00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 0x7f15180a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1524009230 tx=0x7f15240058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:15.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:15.860+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f151c09ca50 con 0x7f1534105180 2024-04-25T15:04:16.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.073+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1534062b80 con 0x7f1534105180 2024-04-25T15:04:16.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.076+0000 7f152be00640 1 -- 172.21.15.19:0/1425369371 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7f151c0a0d50 con 0x7f1534105180 2024-04-25T15:04:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:16.078 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 msgr2=0x7f15180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 0x7f15180a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1524009230 tx=0x7f15240058d0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 msgr2=0x7f1534198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534198560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f151c00e9c0 tx=0x7f151c00ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:16.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 shutdown_connections 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1534107ab0 0x7f153419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15180a3bb0 0x7f15180a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1534105180 0x7f1534198560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 --2- 172.21.15.19:0/1425369371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1534102850 0x7f1534198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 >> 172.21.15.19:0/1425369371 conn(0x7f15340fc220 msgr2=0x7f15341046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 shutdown_connections 2024-04-25T15:04:16.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.078+0000 7f1533e00640 1 -- 172.21.15.19:0/1425369371 wait complete. 2024-04-25T15:04:16.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:04:16.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.314+0000 7f20d2200640 1 -- 172.21.15.19:0/4239178372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 msgr2=0x7f20cc076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.314+0000 7f20d2200640 1 --2- 172.21.15.19:0/4239178372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc076d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009900 tx=0x7f20bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.314+0000 7f20d2200640 1 -- 172.21.15.19:0/4239178372 shutdown_connections 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.314+0000 7f20d2200640 1 --2- 172.21.15.19:0/4239178372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 0x7f20cc078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.314+0000 7f20d2200640 1 --2- 172.21.15.19:0/4239178372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.315+0000 7f20d2200640 1 --2- 172.21.15.19:0/4239178372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc076d30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.315+0000 7f20d2200640 1 -- 172.21.15.19:0/4239178372 >> 172.21.15.19:0/4239178372 conn(0x7f20cc0ff600 msgr2=0x7f20cc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.315+0000 7f20d2200640 1 -- 172.21.15.19:0/4239178372 shutdown_connections 2024-04-25T15:04:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.315+0000 7f20d2200640 1 -- 172.21.15.19:0/4239178372 wait complete. 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 Processor -- start 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 -- start start 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 0x7f20cc1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc113fd0 con 0x7f20cc0784f0 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc113e50 con 0x7f20cc077b50 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.316+0000 7f20d2200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114150 con 0x7f20cc076910 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44290/0 (socket says 172.21.15.19:44290) 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 -- 172.21.15.19:0/3376775974 learned_addr learned my addr 172.21.15.19:0/3376775974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d1800640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 0x7f20cc1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20cbe00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 -- 172.21.15.19:0/3376775974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 msgr2=0x7f20cc19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 -- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 msgr2=0x7f20cc1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 0x7f20cc1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 -- 172.21.15.19:0/3376775974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1a6320 con 0x7f20cc076910 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20cbe00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.317+0000 7f20d0e00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005940 tx=0x7f20bc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.318+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc03f030 con 0x7f20cc076910 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.318+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc003cb0 con 0x7f20cc076910 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.318+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc035070 con 0x7f20cc076910 2024-04-25T15:04:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.318+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc1a65b0 con 0x7f20cc076910 2024-04-25T15:04:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.318+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a6a50 con 0x7f20cc076910 2024-04-25T15:04:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.319+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc110c20 con 0x7f20cc076910 2024-04-25T15:04:16.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.321+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20bc042030 con 0x7f20cc076910 2024-04-25T15:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.324+0000 7f20c9600640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 0x7f20b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.324+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f20bc0f0840 con 0x7f20cc076910 2024-04-25T15:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.324+0000 7f20cbe00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 0x7f20b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.324+0000 7f20cbe00640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 0x7f20b00a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20cc19fc10 tx=0x7f20b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:16.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.327+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20bc0bd1d0 con 0x7f20cc076910 2024-04-25T15:04:16.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.540+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20cc062b50 con 0x7f20cc076910 2024-04-25T15:04:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.542+0000 7f20c9600640 1 -- 172.21.15.19:0/3376775974 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f20bc003e60 con 0x7f20cc076910 2024-04-25T15:04:16.543 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:04:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 msgr2=0x7f20b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 0x7f20b00a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20cc19fc10 tx=0x7f20b4005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 msgr2=0x7f20cc19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005940 tx=0x7f20bc0042f0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 shutdown_connections 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20cc0784f0 0x7f20cc1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f20b00a3c30 0x7f20b00a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20cc077b50 0x7f20cc19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 --2- 172.21.15.19:0/3376775974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20cc076910 0x7f20cc19e750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 >> 172.21.15.19:0/3376775974 conn(0x7f20cc0ff600 msgr2=0x7f20cc100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 shutdown_connections 2024-04-25T15:04:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.544+0000 7f20d2200640 1 -- 172.21.15.19:0/3376775974 wait complete. 2024-04-25T15:04:16.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 -- 172.21.15.19:0/2010136566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368104090 msgr2=0x7f53681044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 --2- 172.21.15.19:0/2010136566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368104090 0x7f53681044b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5358009ca0 tx=0x7f535802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 -- 172.21.15.19:0/2010136566 shutdown_connections 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 --2- 172.21.15.19:0/2010136566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105c40 0x7f536810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 --2- 172.21.15.19:0/2010136566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368105280 0x7f5368105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 --2- 172.21.15.19:0/2010136566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368104090 0x7f53681044b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 -- 172.21.15.19:0/2010136566 >> 172.21.15.19:0/2010136566 conn(0x7f53680ff600 msgr2=0x7f5368101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 -- 172.21.15.19:0/2010136566 shutdown_connections 2024-04-25T15:04:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.774+0000 7f536d000640 1 -- 172.21.15.19:0/2010136566 wait complete. 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.775+0000 7f536d000640 1 Processor -- start 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 -- start start 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368105c40 0x7f5368076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f536810fc20 con 0x7f5368105280 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f536810faa0 con 0x7f5368104090 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f536d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f536810fda0 con 0x7f5368105c40 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5366a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5367400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5367400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56656/0 (socket says 172.21.15.19:56656) 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5367400640 1 -- 172.21.15.19:0/589200452 learned_addr learned my addr 172.21.15.19:0/589200452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5366a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49136/0 (socket says 172.21.15.19:49136) 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.776+0000 7f5367e00640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368105c40 0x7f5368076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 -- 172.21.15.19:0/589200452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368105c40 msgr2=0x7f5368076870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368105c40 0x7f5368076870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 -- 172.21.15.19:0/589200452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 msgr2=0x7f5368079d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 -- 172.21.15.19:0/589200452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5368077130 con 0x7f5368105280 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5367400640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f5366a00640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f535000e9f0 tx=0x7f535000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535000ccc0 con 0x7f5368105280 2024-04-25T15:04:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500108e0 con 0x7f5368105280 2024-04-25T15:04:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681a6700 con 0x7f5368105280 2024-04-25T15:04:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.777+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350018880 con 0x7f5368105280 2024-04-25T15:04:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.778+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a6b60 con 0x7f5368105280 2024-04-25T15:04:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.779+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536810b450 con 0x7f5368105280 2024-04-25T15:04:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.781+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5350010430 con 0x7f5368105280 2024-04-25T15:04:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.782+0000 7f535fe00640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 0x7f534c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.782+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f53500d06c0 con 0x7f5368105280 2024-04-25T15:04:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.782+0000 7f5367400640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 0x7f534c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:16.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.783+0000 7f5367400640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 0x7f534c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5358004ab0 tx=0x7f5358004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:16.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:16.786+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f535009ca00 con 0x7f5368105280 2024-04-25T15:04:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4245451281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1425369371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4245451281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1425369371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4245451281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:16.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1425369371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:17.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.000+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5368062b50 con 0x7f5368105280 2024-04-25T15:04:17.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.001+0000 7f535fe00640 1 -- 172.21.15.19:0/589200452 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f53500a0d00 con 0x7f5368105280 2024-04-25T15:04:17.003 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:04:17.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 msgr2=0x7f534c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 0x7f534c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5358004ab0 tx=0x7f5358004a40 comp rx=0 tx=0).stop 2024-04-25T15:04:17.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 msgr2=0x7f536807a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f535000e9f0 tx=0x7f535000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 shutdown_connections 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368105c40 0x7f5368076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5368105280 0x7f536807a240 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f534c0a3c30 0x7f534c0a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 --2- 172.21.15.19:0/589200452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5368104090 0x7f5368079d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.003+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 >> 172.21.15.19:0/589200452 conn(0x7f53680ff600 msgr2=0x7f5368108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.004+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 shutdown_connections 2024-04-25T15:04:17.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.004+0000 7f536d000640 1 -- 172.21.15.19:0/589200452 wait complete. 2024-04-25T15:04:17.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 -- 172.21.15.19:0/2544542713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c740749b0 msgr2=0x7f4c74074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 --2- 172.21.15.19:0/2544542713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c740749b0 0x7f4c74074e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009900 tx=0x7f4c6002a410 comp rx=0 tx=0).stop 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 -- 172.21.15.19:0/2544542713 shutdown_connections 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 --2- 172.21.15.19:0/2544542713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 0x7f4c7410c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 --2- 172.21.15.19:0/2544542713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c740749b0 0x7f4c74074e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 --2- 172.21.15.19:0/2544542713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c74075f90 0x7f4c740743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 -- 172.21.15.19:0/2544542713 >> 172.21.15.19:0/2544542713 conn(0x7f4c740fd3f0 msgr2=0x7f4c740ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 -- 172.21.15.19:0/2544542713 shutdown_connections 2024-04-25T15:04:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.231+0000 7f4c73e00640 1 -- 172.21.15.19:0/2544542713 wait complete. 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.232+0000 7f4c73e00640 1 Processor -- start 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 -- start start 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c74075f90 0x7f4c7419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 0x7f4c741a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410fb90 con 0x7f4c740749b0 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410fa10 con 0x7f4c74109d20 2024-04-25T15:04:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7410fd10 con 0x7f4c74075f90 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c74075f90 0x7f4c7419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49154/0 (socket says 172.21.15.19:49154) 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 -- 172.21.15.19:0/180333780 learned_addr learned my addr 172.21.15.19:0/180333780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c73400640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 0x7f4c741a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 -- 172.21.15.19:0/180333780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c74075f90 msgr2=0x7f4c7419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c74075f90 0x7f4c7419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 -- 172.21.15.19:0/180333780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 msgr2=0x7f4c741a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 0x7f4c741a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 -- 172.21.15.19:0/180333780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c741a1e60 con 0x7f4c740749b0 2024-04-25T15:04:17.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.234+0000 7f4c72a00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400e9f0 tx=0x7f4c6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:17.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.235+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400cd80 con 0x7f4c740749b0 2024-04-25T15:04:17.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.235+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c640108e0 con 0x7f4c740749b0 2024-04-25T15:04:17.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.235+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64018980 con 0x7f4c740749b0 2024-04-25T15:04:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.235+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c741a2150 con 0x7f4c740749b0 2024-04-25T15:04:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.235+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a2620 con 0x7f4c740749b0 2024-04-25T15:04:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.236+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c74108a50 con 0x7f4c740749b0 2024-04-25T15:04:17.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.238+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c64010430 con 0x7f4c740749b0 2024-04-25T15:04:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.241+0000 7f4c6b400640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 0x7f4c580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.241+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4c640d0710 con 0x7f4c740749b0 2024-04-25T15:04:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.241+0000 7f4c72000640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 0x7f4c580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.241+0000 7f4c72000640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 0x7f4c580a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000c00 tx=0x7f4c600314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.245+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c6409ca50 con 0x7f4c740749b0 2024-04-25T15:04:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.452+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4c74062b50 con 0x7f4c740749b0 2024-04-25T15:04:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.453+0000 7f4c6b400640 1 -- 172.21.15.19:0/180333780 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f4c640a0d50 con 0x7f4c740749b0 2024-04-25T15:04:17.455 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.455+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 msgr2=0x7f4c580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.455+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 0x7f4c580a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000c00 tx=0x7f4c600314a0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 msgr2=0x7f4c7419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400e9f0 tx=0x7f4c6400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 shutdown_connections 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f4c580a3c30 0x7f4c580a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c74109d20 0x7f4c741a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c74075f90 0x7f4c7419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 --2- 172.21.15.19:0/180333780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c740749b0 0x7f4c7419a240 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 >> 172.21.15.19:0/180333780 conn(0x7f4c740fd3f0 msgr2=0x7f4c74105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:17.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 shutdown_connections 2024-04-25T15:04:17.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.456+0000 7f4c73e00640 1 -- 172.21.15.19:0/180333780 wait complete. 2024-04-25T15:04:17.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.689+0000 7f4186400640 1 -- 172.21.15.19:0/229499055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41800ffff0 msgr2=0x7f4180100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.689+0000 7f4186400640 1 --2- 172.21.15.19:0/229499055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41800ffff0 0x7f4180100410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4174009900 tx=0x7f417402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- 172.21.15.19:0/229499055 shutdown_connections 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- 172.21.15.19:0/229499055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4180101720 0x7f4180101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- 172.21.15.19:0/229499055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180068400 0x7f41801011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- 172.21.15.19:0/229499055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41800ffff0 0x7f4180100410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- 172.21.15.19:0/229499055 >> 172.21.15.19:0/229499055 conn(0x7f41800fb580 msgr2=0x7f41800fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- 172.21.15.19:0/229499055 shutdown_connections 2024-04-25T15:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- 172.21.15.19:0/229499055 wait complete. 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 Processor -- start 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- start start 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4180068400 0x7f4180198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101720 0x7f418019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f418010c850 con 0x7f41800ffff0 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f418010c6d0 con 0x7f4180101720 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.690+0000 7f4186400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f418010c9d0 con 0x7f4180068400 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49172/0 (socket says 172.21.15.19:49172) 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 -- 172.21.15.19:0/4049123388 learned_addr learned my addr 172.21.15.19:0/4049123388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f4185000640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4180068400 0x7f4180198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f4185a00640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101720 0x7f418019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 -- 172.21.15.19:0/4049123388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4180068400 msgr2=0x7f4180198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4180068400 0x7f4180198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 -- 172.21.15.19:0/4049123388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101720 msgr2=0x7f418019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101720 0x7f418019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 -- 172.21.15.19:0/4049123388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418019fc30 con 0x7f41800ffff0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.692+0000 7f417fe00640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f417000d770 tx=0x7f417000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.693+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170017070 con 0x7f41800ffff0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.693+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4170009c30 con 0x7f41800ffff0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.693+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418019ff20 con 0x7f41800ffff0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.693+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000f620 con 0x7f41800ffff0 2024-04-25T15:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.693+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a03e0 con 0x7f41800ffff0 2024-04-25T15:04:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.694+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4180108e40 con 0x7f41800ffff0 2024-04-25T15:04:17.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.696+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f417001a070 con 0x7f41800ffff0 2024-04-25T15:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.697+0000 7f417d600640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 0x7f41680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.698+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f41700cfdb0 con 0x7f41800ffff0 2024-04-25T15:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.698+0000 7f4185000640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 0x7f41680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.699+0000 7f4185000640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 0x7f41680a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4174005ab0 tx=0x7f4174005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.700+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4170009de0 con 0x7f41800ffff0 2024-04-25T15:04:17.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:17 smithi083 ceph-mon[32919]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:17.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:17 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3376775974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:17 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/589200452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:17 smithi102 ceph-mon[33005]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:17.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:17 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3376775974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:17 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/589200452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:17 smithi019 ceph-mon[28090]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:17.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:17 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3376775974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:17 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/589200452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:17.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.953+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f418002d5c0 con 0x7f41800ffff0 2024-04-25T15:04:17.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.954+0000 7f417d600640 1 -- 172.21.15.19:0/4049123388 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7f417009c150 con 0x7f41800ffff0 2024-04-25T15:04:17.956 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:17.956 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:04:17.956 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:04:17.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 msgr2=0x7f41680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 0x7f41680a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4174005ab0 tx=0x7f4174005a20 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 msgr2=0x7f4180198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f417000d770 tx=0x7f417000dc30 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 shutdown_connections 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f41680a3bb0 0x7f41680a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101720 0x7f418019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41800ffff0 0x7f4180198550 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 --2- 172.21.15.19:0/4049123388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4180068400 0x7f4180198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.957+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 >> 172.21.15.19:0/4049123388 conn(0x7f41800fb580 msgr2=0x7f4180102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.958+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 shutdown_connections 2024-04-25T15:04:17.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:17.958+0000 7f4186400640 1 -- 172.21.15.19:0/4049123388 wait complete. 2024-04-25T15:04:17.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 -- 172.21.15.19:0/3880757447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83341031b0 msgr2=0x7f8334103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 --2- 172.21.15.19:0/3880757447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83341031b0 0x7f8334103630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 -- 172.21.15.19:0/3880757447 shutdown_connections 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 --2- 172.21.15.19:0/3880757447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8334103b70 0x7f833410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 --2- 172.21.15.19:0/3880757447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83341031b0 0x7f8334103630 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 --2- 172.21.15.19:0/3880757447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 -- 172.21.15.19:0/3880757447 >> 172.21.15.19:0/3880757447 conn(0x7f83340fd530 msgr2=0x7f83340ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 -- 172.21.15.19:0/3880757447 shutdown_connections 2024-04-25T15:04:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.197+0000 7f8339e00640 1 -- 172.21.15.19:0/3880757447 wait complete. 2024-04-25T15:04:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 Processor -- start 2024-04-25T15:04:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 -- start start 2024-04-25T15:04:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 0x7f8334198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f833410db50 con 0x7f8334103b70 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.198+0000 7f8339e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f833410d9d0 con 0x7f83341031b0 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8339e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f833410dcd0 con 0x7f8334101fc0 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49202/0 (socket says 172.21.15.19:49202) 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 -- 172.21.15.19:0/24748530 learned_addr learned my addr 172.21.15.19:0/24748530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8332a00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8332000640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 0x7f8334198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 -- 172.21.15.19:0/24748530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 msgr2=0x7f83341981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 -- 172.21.15.19:0/24748530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 msgr2=0x7f8334198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 0x7f8334198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 -- 172.21.15.19:0/24748530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419fde0 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8332000640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 0x7f8334198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8332a00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.199+0000 7f8333400640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e970 tx=0x7f832c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.200+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00ccc0 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.200+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c00ce60 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.200+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c010430 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.200+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83341a00d0 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.200+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a0620 con 0x7f8334103b70 2024-04-25T15:04:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.201+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334109360 con 0x7f8334103b70 2024-04-25T15:04:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.203+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f832c004ff0 con 0x7f8334103b70 2024-04-25T15:04:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.204+0000 7f8338a00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 0x7f83180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.205+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f832c0d0920 con 0x7f8334103b70 2024-04-25T15:04:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.205+0000 7f8332a00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 0x7f83180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.205+0000 7f8332a00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 0x7f83180a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8334103010 tx=0x7f8324009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.209+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f832c09d2b0 con 0x7f8334103b70 2024-04-25T15:04:18.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.420+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8334062b50 con 0x7f8334103b70 2024-04-25T15:04:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.429+0000 7f8338a00640 1 -- 172.21.15.19:0/24748530 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22580 (secure 0 0 0) 0x7f832c09cbb0 con 0x7f8334103b70 2024-04-25T15:04:18.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:18.431 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:02:57.459165+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:18.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.432+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 msgr2=0x7f83180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.432+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 0x7f83180a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8334103010 tx=0x7f8324009500 comp rx=0 tx=0).stop 2024-04-25T15:04:18.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.432+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 msgr2=0x7f833419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.432+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e970 tx=0x7f832c00ed70 comp rx=0 tx=0).stop 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 shutdown_connections 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334103b70 0x7f833419f770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f83180a3c30 0x7f83180a60f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83341031b0 0x7f8334198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 --2- 172.21.15.19:0/24748530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8334101fc0 0x7f83341981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 >> 172.21.15.19:0/24748530 conn(0x7f83340fd530 msgr2=0x7f8334106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 shutdown_connections 2024-04-25T15:04:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.433+0000 7f8339e00640 1 -- 172.21.15.19:0/24748530 wait complete. 2024-04-25T15:04:18.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T15:02:00.518771+0000', 'modified': '2024-04-25T15:02:04.773178+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24590}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/2613932433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 2613932433}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 2613932433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24590': {'gid': 24590, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/2949462770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2949462770}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2949462770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24590, 'qdb_cluster': [24590]} 2024-04-25T15:04:18.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T15:04:18.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 -- 172.21.15.19:0/2006097796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 msgr2=0x7f7180105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 --2- 172.21.15.19:0/2006097796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f7180105700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7168009ca0 tx=0x7f716802a8b0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 -- 172.21.15.19:0/2006097796 shutdown_connections 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 --2- 172.21.15.19:0/2006097796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 0x7f718010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 --2- 172.21.15.19:0/2006097796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f7180105700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 --2- 172.21.15.19:0/2006097796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 0x7f71801044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 -- 172.21.15.19:0/2006097796 >> 172.21.15.19:0/2006097796 conn(0x7f71800ff600 msgr2=0x7f7180101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 -- 172.21.15.19:0/2006097796 shutdown_connections 2024-04-25T15:04:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.672+0000 7f7186400640 1 -- 172.21.15.19:0/2006097796 wait complete. 2024-04-25T15:04:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 Processor -- start 2024-04-25T15:04:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 -- start start 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 0x7f718019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 0x7f71801a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.673+0000 7f7186400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f718010fc20 con 0x7f7180105280 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f7186400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f718010faa0 con 0x7f7180104090 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f7186400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f718010fda0 con 0x7f7180105c40 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49232/0 (socket says 172.21.15.19:49232) 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 -- 172.21.15.19:0/2651090196 learned_addr learned my addr 172.21.15.19:0/2651090196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f7185a00640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 0x7f71801a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f7185000640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 0x7f718019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 -- 172.21.15.19:0/2651090196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 msgr2=0x7f71801a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 0x7f71801a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 -- 172.21.15.19:0/2651090196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 msgr2=0x7f718019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 0x7f718019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.674+0000 7f717fe00640 1 -- 172.21.15.19:0/2651090196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71801a1ec0 con 0x7f7180105280 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f717fe00640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f716802aea0 tx=0x7f7168003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f717d600640 1 -- 172.21.15.19:0/2651090196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168004450 con 0x7f7180105280 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f717d600640 1 -- 172.21.15.19:0/2651090196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7168040070 con 0x7f7180105280 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71801a2150 con 0x7f7180105280 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f717d600640 1 -- 172.21.15.19:0/2651090196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168045c10 con 0x7f7180105280 2024-04-25T15:04:18.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.675+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a26a0 con 0x7f7180105280 2024-04-25T15:04:18.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.678+0000 7f718a801740 1 Processor -- start 2024-04-25T15:04:18.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.678+0000 7f717d600640 1 -- 172.21.15.19:0/2651090196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7168045470 con 0x7f7180105280 2024-04-25T15:04:18.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 -- start start 2024-04-25T15:04:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b27b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797b1dab10 con 0x55797b276150 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55797b1812b0 con 0x55797b27bcb0 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f718a801740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55797b1d6ad0 con 0x55797b276b40 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f7185a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f7185a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49248/0 (socket says 172.21.15.19:49248) 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f7185a00640 1 -- 172.21.15.19:0/3364905409 learned_addr learned my addr 172.21.15.19:0/3364905409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f717fe00640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.679+0000 7f7185000640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b27b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 -- 172.21.15.19:0/3364905409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 msgr2=0x55797b27b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b27b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 -- 172.21.15.19:0/3364905409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 msgr2=0x55797b27e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 -- 172.21.15.19:0/3364905409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797aeed2d0 con 0x55797b276150 2024-04-25T15:04:18.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185000640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b27b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185a00640 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f717800b700 tx=0x7f717800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b015c8c00c09094f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f717d600640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 0x7f71640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7176000640 1 -- 172.21.15.19:0/3364905409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178003dd0 con 0x55797b276150 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f717d600640 1 -- 172.21.15.19:0/2651090196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f71680f1880 con 0x7f7180105280 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7176000640 1 -- 172.21.15.19:0/3364905409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7178015070 con 0x55797b276150 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7176000640 1 -- 172.21.15.19:0/3364905409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780106a0 con 0x55797b276150 2024-04-25T15:04:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.680+0000 7f7185000640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 0x7f71640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 -- 172.21.15.19:0/3364905409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 msgr2=0x55797b276570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f7185000640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 0x7f71640a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55797b276f40 tx=0x7f7170005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f717800b700 tx=0x7f717800bbc0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 -- 172.21.15.19:0/3364905409 shutdown_connections 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b27b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 --2- 172.21.15.19:0/3364905409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b276570 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 -- 172.21.15.19:0/3364905409 >> 172.21.15.19:0/3364905409 conn(0x55797b2717c0 msgr2=0x55797b271bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 -- 172.21.15.19:0/3364905409 shutdown_connections 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.681+0000 7f718a801740 1 -- 172.21.15.19:0/3364905409 wait complete. 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.682+0000 7f718a801740 1 -- start start 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.682+0000 7f718a801740 1 Processor -- start 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b287910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797b27ec60 con 0x55797b276150 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55797b194570 con 0x55797b27bcb0 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55797b27ede0 con 0x55797b276b40 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f718a801740 10 client.0 _ll_register_callbacks cb 0x55797b162fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49250/0 (socket says 172.21.15.19:49250) 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 -- 172.21.15.19:0/1964274981 learned_addr learned my addr 172.21.15.19:0/1964274981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f717fe00640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185a00640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b287910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 -- 172.21.15.19:0/1964274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 msgr2=0x55797b2808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 -- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 msgr2=0x55797b287910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b287910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.683+0000 7f7185000640 1 -- 172.21.15.19:0/1964274981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797afb76b0 con 0x55797b276150 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f7185000640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f717001e9d0 tx=0x7f7170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3d4c3927e475d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f716ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170009db0 con 0x55797b276150 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f7170018030 con 0x55797b276150 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7178015070 con 0x55797b276150 2024-04-25T15:04:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7170004030 con 0x55797b276150 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f718a801740 10 client.15000 fetch_fsmap learned FSMap version 20 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.684+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55797b28ada0 con 0x55797b276150 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7178015070 con 0x55797b276150 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f717002a430 con 0x55797b276150 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 10 client.15000 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 10 client.15000 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 20 client.15000 populate_metadata read hostname 'smithi019' 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b28d400 0x55797b28e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 4 client.15000 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55797b1e8530 con 0x55797b28d400 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.685+0000 7f717fe00640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b28d400 0x55797b28e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.686+0000 7f717fe00640 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b28d400 0x55797b28e870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0faae08ef87ac54 server_cookie=29ccb7815714bc76 in_seq=0 out_seq=0 2024-04-25T15:04:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.686+0000 7f716ea00640 10 client.15000 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f716ea00640 1 -- 172.21.15.19:0/1964274981 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55797b1e8530 con 0x55797b28d400 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f716ea00640 10 client.15000 handle_command_reply: tid=0 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 client.15000 shutdown 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 msgr2=0x55797b280390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f717001e9d0 tx=0x7f7170004730 comp rx=0 tx=0).stop 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 shutdown_connections 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b28d400 0x55797b28e870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b287910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 --2- 172.21.15.19:0/1964274981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b280390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f716ea00640 0 client.15000 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 >> 172.21.15.19:0/1964274981 conn(0x55797b2717c0 msgr2=0x55797b270a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 shutdown_connections 2024-04-25T15:04:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.687+0000 7f718a801740 1 -- 172.21.15.19:0/1964274981 wait complete. 2024-04-25T15:04:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.688+0000 7f718a801740 20 client.15000 trim_cache size 0 max 16384 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 Processor -- start 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 -- start start 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797b28cbb0 con 0x55797b276150 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55797b28d280 con 0x55797b27bcb0 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f718a801740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55797b28d950 con 0x55797b276b40 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49262/0 (socket says 172.21.15.19:49262) 2024-04-25T15:04:18.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f7185000640 1 -- 172.21.15.19:0/1259476037 learned_addr learned my addr 172.21.15.19:0/1259476037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.708+0000 7f7185a00640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f717fe00640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 -- 172.21.15.19:0/1259476037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 msgr2=0x55797b2878d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2878d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 -- 172.21.15.19:0/1259476037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 msgr2=0x55797b27e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 -- 172.21.15.19:0/1259476037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797b26bef0 con 0x55797b276150 2024-04-25T15:04:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185a00640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7185000640 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7170019040 tx=0x7f717001ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20411a75235c2934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7175600640 1 -- 172.21.15.19:0/1259476037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717002ab10 con 0x55797b276150 2024-04-25T15:04:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7175600640 1 -- 172.21.15.19:0/1259476037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7170030030 con 0x55797b276150 2024-04-25T15:04:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.709+0000 7f7175600640 1 -- 172.21.15.19:0/1259476037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170004430 con 0x55797b276150 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 -- 172.21.15.19:0/1259476037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 msgr2=0x55797b271260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7170019040 tx=0x7f717001ee90 comp rx=0 tx=0).stop 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 -- 172.21.15.19:0/1259476037 shutdown_connections 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b27bcb0 0x55797b27e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b276b40 0x55797b2878d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 --2- 172.21.15.19:0/1259476037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b271260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 -- 172.21.15.19:0/1259476037 >> 172.21.15.19:0/1259476037 conn(0x55797b2717c0 msgr2=0x55797b271ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 -- 172.21.15.19:0/1259476037 shutdown_connections 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.710+0000 7f718a801740 1 -- 172.21.15.19:0/1259476037 wait complete. 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 -- start start 2024-04-25T15:04:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 Processor -- start 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b282d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b27bcb0 0x55797b286280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55797b28e6f0 con 0x55797b276150 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55797b28e020 con 0x55797b276b40 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.711+0000 7f718a801740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55797b28edc0 con 0x55797b27bcb0 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f718a801740 10 client.0 _ll_register_callbacks cb 0x7f717001e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b282d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56778/0 (socket says 172.21.15.19:56778) 2024-04-25T15:04:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 -- 172.21.15.19:0/2482134790 learned_addr learned my addr 172.21.15.19:0/2482134790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f7185000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b282d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49266/0 (socket says 172.21.15.19:49266) 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f7185a00640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b27bcb0 0x55797b286280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 -- 172.21.15.19:0/2482134790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b27bcb0 msgr2=0x55797b286280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b27bcb0 0x55797b286280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 -- 172.21.15.19:0/2482134790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 msgr2=0x55797b282d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b282d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 -- 172.21.15.19:0/2482134790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797b286870 con 0x55797b276b40 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f717fe00640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71680bdaa0 tx=0x7f71680bd530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09d411e85ca30ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f716ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.712+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71680be210 con 0x55797b276b40 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7170030030 con 0x55797b276b40 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f71680bbe10 con 0x55797b276b40 2024-04-25T15:04:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71680bdb10 con 0x55797b276b40 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 10 client.25112 fetch_fsmap learned FSMap version 20 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55797b287b70 con 0x55797b276b40 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7170030030 con 0x55797b276b40 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f71680d9070 con 0x55797b276b40 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 10 client.25112 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 10 client.25112 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 20 client.25112 populate_metadata read hostname 'smithi019' 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b2f7cf0 0x55797b2f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 4 client.25112 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T15:04:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.713+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "status"}) -- 0x55797b1e23a0 con 0x55797b2f7cf0 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f7185000640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b2f7cf0 0x55797b2f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f7185000640 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b2f7cf0 0x55797b2f6a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1392022f5f7f618 server_cookie=75669f3d7328410 in_seq=0 out_seq=0 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f716ea00640 10 client.25112 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f716ea00640 1 -- 172.21.15.19:0/2482134790 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55797b1e23a0 con 0x55797b2f7cf0 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f716ea00640 10 client.25112 handle_command_reply: tid=0 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.714+0000 7f718a801740 1 client.25112 shutdown 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 msgr2=0x55797b2832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71680bdaa0 tx=0x7f71680bd530 comp rx=0 tx=0).stop 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 shutdown_connections 2024-04-25T15:04:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x55797b2f7cf0 0x55797b2f6a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55797b27bcb0 0x55797b286280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55797b276b40 0x55797b2832d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f716ea00640 0 client.25112 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 --2- 172.21.15.19:0/2482134790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55797b276150 0x55797b282d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 >> 172.21.15.19:0/2482134790 conn(0x55797b2717c0 msgr2=0x55797b277d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 shutdown_connections 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 1 -- 172.21.15.19:0/2482134790 wait complete. 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.715+0000 7f718a801740 20 client.25112 trim_cache size 0 max 16384 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:04:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "42f950da-0313-11ef-bc93-c7b262605968", 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24590, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 137.66959466099999, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 20, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 90, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 145.29815504300001 2024-04-25T15:04:18.722 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 msgr2=0x7f71640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 0x7f71640a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55797b276f40 tx=0x7f7170005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 msgr2=0x7f718019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f716802aea0 tx=0x7f7168003c20 comp rx=0 tx=0).stop 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 shutdown_connections 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7180105c40 0x7f71801a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7180105280 0x7f718019a7e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f71640a3bb0 0x7f71640a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 --2- 172.21.15.19:0/2651090196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7180104090 0x7f718019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 >> 172.21.15.19:0/2651090196 conn(0x7f71800ff600 msgr2=0x7f71801085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 shutdown_connections 2024-04-25T15:04:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.716+0000 7f7186400640 1 -- 172.21.15.19:0/2651090196 wait complete. 2024-04-25T15:04:18.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/180333780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:18.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4049123388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:18.735 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T15:04:18.746 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T15:04:18.746 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T15:04:18.746 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T15:04:18.746 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T15:04:18.747 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:04:18.747 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:18.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T15:04:18.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/180333780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:18.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4049123388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/180333780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4049123388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.945+0000 7f553be00640 1 -- 172.21.15.19:0/623129250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c075f90 msgr2=0x7f553c0743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.945+0000 7f553be00640 1 --2- 172.21.15.19:0/623129250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c075f90 0x7f553c0743e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f552c009900 tx=0x7f552c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 -- 172.21.15.19:0/623129250 shutdown_connections 2024-04-25T15:04:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 --2- 172.21.15.19:0/623129250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c10e3a0 0x7f553c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 --2- 172.21.15.19:0/623129250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 0x7f553c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 --2- 172.21.15.19:0/623129250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c075f90 0x7f553c0743e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 -- 172.21.15.19:0/623129250 >> 172.21.15.19:0/623129250 conn(0x7f553c0fd360 msgr2=0x7f553c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 -- 172.21.15.19:0/623129250 shutdown_connections 2024-04-25T15:04:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.946+0000 7f553be00640 1 -- 172.21.15.19:0/623129250 wait complete. 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 Processor -- start 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 -- start start 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 0x7f553c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c10e3a0 0x7f553c1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f553c113ed0 con 0x7f553c075f90 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f553c113d50 con 0x7f553c0749b0 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f553be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f553c114050 con 0x7f553c10e3a0 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f5533e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f5533e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49286/0 (socket says 172.21.15.19:49286) 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.947+0000 7f5533e00640 1 -- 172.21.15.19:0/4160476417 learned_addr learned my addr 172.21.15.19:0/4160476417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f553aa00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 0x7f553c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f553b400640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c10e3a0 0x7f553c1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 -- 172.21.15.19:0/4160476417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c10e3a0 msgr2=0x7f553c1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c10e3a0 0x7f553c1a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 -- 172.21.15.19:0/4160476417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 msgr2=0x7f553c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 0x7f553c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 -- 172.21.15.19:0/4160476417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c1a61e0 con 0x7f553c075f90 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533e00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f552400e9f0 tx=0x7f552400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400cd80 con 0x7f553c075f90 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55240108e0 con 0x7f553c075f90 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.948+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a64d0 con 0x7f553c075f90 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.949+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524018980 con 0x7f553c075f90 2024-04-25T15:04:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.949+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a6a20 con 0x7f553c075f90 2024-04-25T15:04:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.950+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c104570 con 0x7f553c075f90 2024-04-25T15:04:18.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.952+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5524010430 con 0x7f553c075f90 2024-04-25T15:04:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.953+0000 7f5533400640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 0x7f55200a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.953+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f55240d07d0 con 0x7f553c075f90 2024-04-25T15:04:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.954+0000 7f553aa00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 0x7f55200a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:18.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.954+0000 7f553aa00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 0x7f55200a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f552c006010 tx=0x7f552c031430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:18.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:18.958+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f552409d160 con 0x7f553c075f90 2024-04-25T15:04:19.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.220+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f553c062b50 con 0x7f553c075f90 2024-04-25T15:04:19.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.448+0000 7f5533400640 1 -- 172.21.15.19:0/4160476417 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f552409ca60 con 0x7f553c075f90 2024-04-25T15:04:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.452+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 msgr2=0x7f55200a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.452+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 0x7f55200a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f552c006010 tx=0x7f552c031430 comp rx=0 tx=0).stop 2024-04-25T15:04:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.452+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 msgr2=0x7f553c19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.452+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f552400e9f0 tx=0x7f552400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 shutdown_connections 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c10e3a0 0x7f553c1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f553c075f90 0x7f553c19eb00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55200a3ba0 0x7f55200a6060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 --2- 172.21.15.19:0/4160476417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553c0749b0 0x7f553c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.453+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 >> 172.21.15.19:0/4160476417 conn(0x7f553c0fd360 msgr2=0x7f553c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:19.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.454+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 shutdown_connections 2024-04-25T15:04:19.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.455+0000 7f553be00640 1 -- 172.21.15.19:0/4160476417 wait complete. 2024-04-25T15:04:19.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T15:04:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.667+0000 7f009de00640 1 -- 172.21.15.19:0/2553281688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098107d40 msgr2=0x7f009810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.667+0000 7f009de00640 1 --2- 172.21.15.19:0/2553281688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098107d40 0x7f009810a190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0090009ca0 tx=0x7f009002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 -- 172.21.15.19:0/2553281688 shutdown_connections 2024-04-25T15:04:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 --2- 172.21.15.19:0/2553281688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098107d40 0x7f009810a190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 --2- 172.21.15.19:0/2553281688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 --2- 172.21.15.19:0/2553281688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098102ae0 0x7f0098104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 -- 172.21.15.19:0/2553281688 >> 172.21.15.19:0/2553281688 conn(0x7f00980fc4d0 msgr2=0x7f00980fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 -- 172.21.15.19:0/2553281688 shutdown_connections 2024-04-25T15:04:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.668+0000 7f009de00640 1 -- 172.21.15.19:0/2553281688 wait complete. 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 Processor -- start 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 -- start start 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 0x7f0098102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 0x7f0098100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f009810db80 con 0x7f0098102ae0 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f009810da00 con 0x7f0098105410 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.669+0000 7f009de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f009810dd00 con 0x7f0098107d40 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f0096a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 0x7f0098102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f0097400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 0x7f0098100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f0097400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 0x7f0098100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44454/0 (socket says 172.21.15.19:44454) 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f0096a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 0x7f0098102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49310/0 (socket says 172.21.15.19:49310) 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56810/0 (socket says 172.21.15.19:56810) 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f0097400640 1 -- 172.21.15.19:0/2061294568 learned_addr learned my addr 172.21.15.19:0/2061294568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 msgr2=0x7f0098100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 0x7f0098100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 msgr2=0x7f0098102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 0x7f0098102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 -- 172.21.15.19:0/2061294568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0098101860 con 0x7f0098105410 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f008e000640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9c0 tx=0x7f008000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.670+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000ccc0 con 0x7f0098105410 2024-04-25T15:04:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.671+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00981a21d0 con 0x7f0098105410 2024-04-25T15:04:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.671+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00800108e0 con 0x7f0098105410 2024-04-25T15:04:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.671+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800187a0 con 0x7f0098105410 2024-04-25T15:04:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.671+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981a2770 con 0x7f0098105410 2024-04-25T15:04:19.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.672+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00980fdfc0 con 0x7f0098105410 2024-04-25T15:04:19.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.674+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0080010430 con 0x7f0098105410 2024-04-25T15:04:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.676+0000 7f009ca00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.676+0000 7f0096a00640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 msgr2=0x7f00780a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.676+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f00800d0740 con 0x7f0098105410 2024-04-25T15:04:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.676+0000 7f0096a00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:04:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.680+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f008009ca80 con 0x7f0098105410 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-mon[32919]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/24748530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: ignoring --setuser ceph since I am not root 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: ignoring --setgroup ceph since I am not root 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-mgr[34327]: -- 172.21.15.83:0/1462113290 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff3da905a0 con 0x55ff3da64800 2024-04-25T15:04:19.692 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:19.630+0000 7f22f509b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:04:19.777 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setuser ceph since I am not root 2024-04-25T15:04:19.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: ignoring --setgroup ceph since I am not root 2024-04-25T15:04:19.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-mon[28090]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:19.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/24748530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:19.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:04:19.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-mon[33005]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:19.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/24748530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:19.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:04:19.808 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: ignoring --setuser ceph since I am not root 2024-04-25T15:04:19.808 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: ignoring --setgroup ceph since I am not root 2024-04-25T15:04:19.808 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-mgr[34885]: -- 172.21.15.102:0/311390376 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580aa5905a0 con 0x5580aa564c00 2024-04-25T15:04:19.808 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:19.627+0000 7fa4a02bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:04:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.877+0000 7f0096a00640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 msgr2=0x7f00780a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:19.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.877+0000 7f0096a00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:04:19.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.887+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f009802d5c0 con 0x7f0098105410 2024-04-25T15:04:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.894+0000 7f009ca00640 1 -- 172.21.15.19:0/2061294568 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f00800a0d80 con 0x7f0098105410 2024-04-25T15:04:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 msgr2=0x7f00780a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:04:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 msgr2=0x7f0098100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f008000e9c0 tx=0x7f008000ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 shutdown_connections 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0098107d40 0x7f0098100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0098105410 0x7f0098100a90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 --2- 172.21.15.19:0/2061294568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0098102ae0 0x7f0098102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 >> 172.21.15.19:0/2061294568 conn(0x7f00980fc4d0 msgr2=0x7f0098103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 shutdown_connections 2024-04-25T15:04:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:19.896+0000 7f009de00640 1 -- 172.21.15.19:0/2061294568 wait complete. 2024-04-25T15:04:19.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T15:04:20.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:19.814+0000 7f22f509b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:20.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:19.917+0000 7f22f509b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 -- 172.21.15.19:0/3557471392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234102850 msgr2=0x7f4234104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 --2- 172.21.15.19:0/3557471392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234102850 0x7f4234104c40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4224009900 tx=0x7f422402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 -- 172.21.15.19:0/3557471392 shutdown_connections 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 --2- 172.21.15.19:0/3557471392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234107ab0 0x7f4234109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 --2- 172.21.15.19:0/3557471392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234105180 0x7f4234107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 --2- 172.21.15.19:0/3557471392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234102850 0x7f4234104c40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 -- 172.21.15.19:0/3557471392 >> 172.21.15.19:0/3557471392 conn(0x7f42340fc240 msgr2=0x7f42340fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 -- 172.21.15.19:0/3557471392 shutdown_connections 2024-04-25T15:04:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.100+0000 7f4233200640 1 -- 172.21.15.19:0/3557471392 wait complete. 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.101+0000 7f4233200640 1 Processor -- start 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 -- start start 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234105180 0x7f423419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234107ab0 0x7f42341a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f423410d900 con 0x7f4234105180 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f423410d780 con 0x7f4234107ab0 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4233200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f423410da80 con 0x7f4234102850 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44472/0 (socket says 172.21.15.19:44472) 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 -- 172.21.15.19:0/1708795977 learned_addr learned my addr 172.21.15.19:0/1708795977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231400640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234105180 0x7f423419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4232800640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234107ab0 0x7f42341a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234107ab0 msgr2=0x7f42341a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234107ab0 0x7f42341a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234105180 msgr2=0x7f423419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.102+0000 7f4231e00640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234105180 0x7f423419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f4231e00640 1 -- 172.21.15.19:0/1708795977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42341a1e70 con 0x7f4234102850 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f4231e00640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4224005940 tx=0x7f42240042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422403f030 con 0x7f4234102850 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4224003cb0 con 0x7f4234102850 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42341a2100 con 0x7f4234102850 2024-04-25T15:04:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224035070 con 0x7f4234102850 2024-04-25T15:04:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.103+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42341a25f0 con 0x7f4234102850 2024-04-25T15:04:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.105+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4234109600 con 0x7f4234102850 2024-04-25T15:04:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.106+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4224042030 con 0x7f4234102850 2024-04-25T15:04:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.107+0000 7f422aa00640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 0x7f42180a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.108+0000 7f4231400640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 msgr2=0x7f42180a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.108+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f42240f0860 con 0x7f4234102850 2024-04-25T15:04:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.108+0000 7f4231400640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 0x7f42180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:04:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.112+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4224003e60 con 0x7f4234102850 2024-04-25T15:04:20.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:19.776+0000 7f4a504a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:04:20.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:20.024+0000 7f4a504a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:20.222 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:19.810+0000 7fa4a02bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:20.223 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:19.914+0000 7fa4a02bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:04:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.309+0000 7f4231400640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 msgr2=0x7f42180a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:20.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.309+0000 7f4231400640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 0x7f42180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:04:20.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.319+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4234062b50 con 0x7f4234102850 2024-04-25T15:04:20.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.326+0000 7f422aa00640 1 -- 172.21.15.19:0/1708795977 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f42240bcba0 con 0x7f4234102850 2024-04-25T15:04:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 msgr2=0x7f42180a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:04:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 0x7f42180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 msgr2=0x7f423419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4224005940 tx=0x7f42240042f0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 shutdown_connections 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f42180a3bf0 0x7f42180a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4234107ab0 0x7f42341a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4234105180 0x7f423419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 --2- 172.21.15.19:0/1708795977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4234102850 0x7f423419a250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 >> 172.21.15.19:0/1708795977 conn(0x7f42340fc240 msgr2=0x7f42341020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 shutdown_connections 2024-04-25T15:04:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:20.330+0000 7f4233200640 1 -- 172.21.15.19:0/1708795977 wait complete. 2024-04-25T15:04:20.357 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T15:04:20.383 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T15:04:20.383 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T15:04:20.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:20.384 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T15:04:20.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:20.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T15:04:20.451 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:20.173+0000 7f4a504a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:04:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T15:04:20.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:04:20.519 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T15:04:20.519 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T15:04:20.519 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:04:20.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T15:04:20.540 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:20.226+0000 7f22f509b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:04:20.540 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:20.310+0000 7f22f509b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:04:20.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:04:20.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-mon[32919]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T15:04:20.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:20 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-mon[33005]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:20.221+0000 7fa4a02bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:04:20.557 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:20.306+0000 7fa4a02bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:04:20.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4160476417' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:04:20.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-mon[28090]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T15:04:20.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.727 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' 2024-04-25T15:04:20.727 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:20.603+0000 7f4a504a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:04:20.727 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:20.724+0000 7f4a504a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:04:21.008 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:21.791 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:21.519+0000 7f22f509b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:04:21.791 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:21.694+0000 7f22f509b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:21.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:21.483+0000 7fa4a02bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:04:21.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:21.652+0000 7fa4a02bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:22.109 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:21.821+0000 7fa4a02bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:04:22.161 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:21.869+0000 7f22f509b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:04:22.161 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:22.159+0000 7f22f509b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:04:22.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:22.108+0000 7fa4a02bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:04:22.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:22.198+0000 7fa4a02bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:04:22.454 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:22.251+0000 7f22f509b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:04:22.625 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:22.381+0000 7f4a504a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:04:22.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:22.453+0000 7f22f509b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:04:22.791 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:22.645+0000 7f22f509b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:22.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:22.401+0000 7fa4a02bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:04:22.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:22.592+0000 7fa4a02bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:22.902 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:22.623+0000 7f4a504a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:22.902 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:22.847+0000 7f4a504a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:04:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 -- 172.21.15.19:0/3871959109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f810f4f0 msgr2=0x7f55f81118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 --2- 172.21.15.19:0/3871959109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f810f4f0 0x7f55f81118e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b0a0 tx=0x7f55f000b3b0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 -- 172.21.15.19:0/3871959109 shutdown_connections 2024-04-25T15:04:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 --2- 172.21.15.19:0/3871959109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f810f4f0 0x7f55f81118e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 --2- 172.21.15.19:0/3871959109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f806a750 0x7f55f806ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 --2- 172.21.15.19:0/3871959109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.224+0000 7f55fe800700 1 -- 172.21.15.19:0/3871959109 >> 172.21.15.19:0/3871959109 conn(0x7f55f80788f0 msgr2=0x7f55f8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.225+0000 7f55fe800700 1 -- 172.21.15.19:0/3871959109 shutdown_connections 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.225+0000 7f55fe800700 1 -- 172.21.15.19:0/3871959109 wait complete. 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.226+0000 7f55fe800700 1 Processor -- start 2024-04-25T15:04:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.226+0000 7f55fe800700 1 -- start start 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 0x7f55f819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8114e40 con 0x7f55f810f4f0 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8114cc0 con 0x7f55f806a750 2024-04-25T15:04:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55fe800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8114fc0 con 0x7f55f806b070 2024-04-25T15:04:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55f7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55f6a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55f7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:56860/0 (socket says 172.21.15.19:56860) 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55f7400700 1 -- 172.21.15.19:0/3897946563 learned_addr learned my addr 172.21.15.19:0/3897946563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.227+0000 7f55f7e00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 0x7f55f819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 msgr2=0x7f55f819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 msgr2=0x7f55f819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 0x7f55f819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 -- 172.21.15.19:0/3897946563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f81a0330 con 0x7f55f806a750 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f6a00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7e00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 0x7f55f819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55f7400700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55e800eb30 tx=0x7f55e800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800cc90 con 0x7f55f806a750 2024-04-25T15:04:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e80044d0 con 0x7f55f806a750 2024-04-25T15:04:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8010430 con 0x7f55f806a750 2024-04-25T15:04:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f81a0620 con 0x7f55f806a750 2024-04-25T15:04:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.228+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a7ea0 con 0x7f55f806a750 2024-04-25T15:04:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.229+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f8069730 con 0x7f55f806a750 2024-04-25T15:04:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.232+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f55e8003ea0 con 0x7f55f806a750 2024-04-25T15:04:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.233+0000 7f55efe00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 0x7f55dc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.233+0000 7f55f6a00700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 msgr2=0x7f55dc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.233+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f55e80d07e0 con 0x7f55f806a750 2024-04-25T15:04:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.233+0000 7f55f6a00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 0x7f55dc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:04:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.235+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55e809d170 con 0x7f55f806a750 2024-04-25T15:04:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.434+0000 7f55f6a00700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 msgr2=0x7f55dc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:23.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.434+0000 7f55f6a00700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 0x7f55dc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:04:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.519+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f55f802df80 con 0x7f55f806a750 2024-04-25T15:04:23.521 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:23.261+0000 7f4a504a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:04:23.521 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:23.389+0000 7f4a504a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:04:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.520+0000 7f55efe00700 1 -- 172.21.15.19:0/3897946563 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f55e809ca70 con 0x7f55f806a750 2024-04-25T15:04:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.523+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 msgr2=0x7f55dc0a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.523+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 0x7f55dc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.523+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 msgr2=0x7f55f819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.523+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55e800eb30 tx=0x7f55e800ee40 comp rx=0 tx=0).stop 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 shutdown_connections 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f55dc0a3d10 0x7f55dc0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f810f4f0 0x7f55f819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f806a750 0x7f55f819b370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 --2- 172.21.15.19:0/3897946563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55f806b070 0x7f55f819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:23.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 >> 172.21.15.19:0/3897946563 conn(0x7f55f80788f0 msgr2=0x7f55f8110940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:23.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 shutdown_connections 2024-04-25T15:04:23.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:23.524+0000 7f55fe800700 1 -- 172.21.15.19:0/3897946563 wait complete. 2024-04-25T15:04:23.540 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:23.240+0000 7f22f509b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:04:23.557 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:23.186+0000 7fa4a02bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:04:23.871 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:23.657+0000 7f4a504a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:04:23.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:23 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3897946563' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:04:23.910 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:23 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3897946563' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:04:23.910 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:23.613+0000 7fa4a02bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:04:23.910 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:23.708+0000 7fa4a02bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:04:23.960 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:23 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3897946563' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:04:23.960 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:23.662+0000 7f22f509b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:04:23.960 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:23.758+0000 7f22f509b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:04:23.960 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:23.959+0000 7f22f509b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:04:24.226 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:23.935+0000 7f4a504a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:04:24.307 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:23.909+0000 7fa4a02bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:04:24.355 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T15:04:24.355 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T15:04:24.356 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:04:24.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:24.356 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:04:24.407 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41939sec preferred_lft 41939sec 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:04:24.408 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:04:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T15:04:24.409 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T15:04:24.410 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T15:04:24.410 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T15:04:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:24.763 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:24.431+0000 7fa4a02bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:04:24.764 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:24.522+0000 7fa4a02bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:04:24.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:24.482+0000 7f22f509b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:04:24.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:24.573+0000 7f22f509b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:04:24.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:24.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:24.860 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T15:04:24.923 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T15:04:24.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:24.927 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T15:04:24.986 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T15:04:24.986 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T15:04:24.986 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T15:04:24.986 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T15:04:24.987 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T15:04:24.987 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T15:04:25.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:24.762+0000 7fa4a02bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:04:25.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:24.893+0000 7fa4a02bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:04:25.057 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:24.991+0000 7fa4a02bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:04:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:25.094 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:24.793+0000 7f4a504a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:04:25.158 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:24.812+0000 7f22f509b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:04:25.158 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:24.944+0000 7f22f509b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:04:25.158 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:25.042+0000 7f22f509b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:04:25.158 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:25.157+0000 7f22f509b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:25.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:25.108+0000 7fa4a02bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:25.622 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:25.395+0000 7f4a504a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:04:25.622 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:25.531+0000 7f4a504a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:04:25.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:25.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:25.782 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T15:04:25.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:25.457+0000 7f22f509b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:04:25.807 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:25.405+0000 7fa4a02bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:04:25.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:25.826 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T15:04:25.916 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:04:25.916 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T15:04:25.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:04:25.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T15:04:25.916 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T15:04:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:26.028 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:25.825+0000 7f4a504a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:04:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:26.082 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T15:04:26.082 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:04:26.082 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T15:04:26.083 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T15:04:26.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:26.295 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:26.006+0000 7fa4a02bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:04:26.295 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z[34881]: 2024-04-25T15:04:26.220+0000 7fa4a02bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:04:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:26.383 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:04:26.383 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T15:04:26.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T15:04:26.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T15:04:26.383 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T15:04:26.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:26.055+0000 7f22f509b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y[34323]: 2024-04-25T15:04:26.269+0000 7f22f509b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: Standby manager daemon z restarted 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: Standby manager daemon z started 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: Standby manager daemon y restarted 2024-04-25T15:04:26.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:04:26.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: Standby manager daemon y started 2024-04-25T15:04:26.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:26 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.550 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:26.538+0000 7f4a504a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: Standby manager daemon z restarted 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: Standby manager daemon z started 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:26.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: Standby manager daemon y restarted 2024-04-25T15:04:26.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:04:26.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: Standby manager daemon y started 2024-04-25T15:04:26.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: Standby manager daemon z restarted 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: Standby manager daemon z started 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.102:0/1974644121' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:26.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: Standby manager daemon y restarted 2024-04-25T15:04:26.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:04:26.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: Standby manager daemon y started 2024-04-25T15:04:26.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:26 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:04:26.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:26.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:04:26.569 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T15:04:26.610 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:04:26.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:04:26.610 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:04:26.699 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T15:04:26.699 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T15:04:26.699 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/94ee5623-1295-4128-a2e6-2e281e9fd96d 2024-04-25T15:04:26.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:26.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/94ee5623-1295-4128-a2e6-2e281e9fd96d /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:04:26.810 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:26.669+0000 7f4a504a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:04:26.881 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:04:26.881 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T15:04:27.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:27.055+0000 7f4a504a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:04:27.384 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:27.248+0000 7f4a504a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:04:27.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:04:27.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-mon[28090]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:27.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-mon[28090]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T15:04:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:27 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:04:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:27 smithi102 ceph-mon[33005]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:27.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:27 smithi102 ceph-mon[33005]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T15:04:27.637 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:27.383+0000 7f4a504a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:04:27.637 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:27.542+0000 7f4a504a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:04:27.661 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:04:27.661 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T15:04:27.661 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.83:3300,172.21.15.102:3300" 2024-04-25T15:04:27.661 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@42f950da-0313-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/94ee5623-1295-4128-a2e6-2e281e9fd96d 2024-04-25T15:04:27.662 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.19:3300/172.21.15.83:3300/172.21.15.102:3300" will pass to kernel 2024-04-25T15:04:27.665 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T15:04:27.665 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T15:04:27.665 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T15:04:27.665 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T15:04:27.665 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T15:04:27.669 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T15:04:27.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:04:27.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:04:27.733 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:04:27.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:27 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:04:27.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:27 smithi083 ceph-mon[32919]: from='mgr.? 172.21.15.83:0/616011695' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:04:27.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:27 smithi083 ceph-mon[32919]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T15:04:27.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:04:27.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:27.848 INFO:teuthology.orchestra.run.smithi019.stdout:client25174 2024-04-25T15:04:27.848 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T15:04:27.849 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:04:27.896 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:04:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:04:28.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:28.059 INFO:teuthology.orchestra.run.smithi019.stdout:client25174 2024-04-25T15:04:28.059 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:04:28.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:04:28.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:28.166 INFO:teuthology.orchestra.run.smithi019.stdout:client25174 2024-04-25T15:04:28.166 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/42f950da-0313-11ef-bc93-c7b262605968.client25174/status') 2024-04-25T15:04:28.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:04:28.288 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:27.939+0000 7f4a504a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:04:28.295 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.25174 (3)172.21.15.19:0/2297875498 2024-04-25T15:04:28.296 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T15:04:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T15:04:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T15:04:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 7.8347e-05 s, 906 kB/s 2024-04-25T15:04:28.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:04:28.308 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T15:04:28.323 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T15:04:28.332 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:04:28.332 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:28.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.528+0000 7f15c3200640 1 -- 172.21.15.19:0/3136865827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 msgr2=0x7f15c410a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.528+0000 7f15c3200640 1 --2- 172.21.15.19:0/3136865827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 0x7f15c410a220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009900 tx=0x7f15bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 -- 172.21.15.19:0/3136865827 shutdown_connections 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 --2- 172.21.15.19:0/3136865827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 0x7f15c410a220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 --2- 172.21.15.19:0/3136865827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4102fd0 0x7f15c4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 --2- 172.21.15.19:0/3136865827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4101de0 0x7f15c4102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 -- 172.21.15.19:0/3136865827 >> 172.21.15.19:0/3136865827 conn(0x7f15c40fd350 msgr2=0x7f15c40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 -- 172.21.15.19:0/3136865827 shutdown_connections 2024-04-25T15:04:28.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.529+0000 7f15c3200640 1 -- 172.21.15.19:0/3136865827 wait complete. 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 Processor -- start 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 -- start start 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 0x7f15c419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4102fd0 0x7f15c419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 0x7f15c41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410d960 con 0x7f15c4101de0 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410d7e0 con 0x7f15c4102fd0 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.530+0000 7f15c3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410dae0 con 0x7f15c4103990 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 0x7f15c419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15c2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 0x7f15c41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4102fd0 0x7f15c419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 0x7f15c419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34972/0 (socket says 172.21.15.19:34972) 2024-04-25T15:04:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15c1e00640 1 -- 172.21.15.19:0/2844746300 learned_addr learned my addr 172.21.15.19:0/2844746300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 msgr2=0x7f15c41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4103990 0x7f15c41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 msgr2=0x7f15c419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 0x7f15c419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15b9400640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41a1dc0 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.531+0000 7f15c1e00640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15c4101de0 0x7f15c419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15b9400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4102fd0 0x7f15c419a6e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e9f0 tx=0x7f15ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00ccc0 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c41a20b0 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac0108e0 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac018880 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.532+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c41a2650 con 0x7f15c4102fd0 2024-04-25T15:04:28.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.533+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c41a2970 con 0x7f15c4102fd0 2024-04-25T15:04:28.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.537+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f15ac010430 con 0x7f15c4102fd0 2024-04-25T15:04:28.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.539+0000 7f15bb400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 0x7f15a80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:28.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.539+0000 7f15c1e00640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 msgr2=0x7f15a80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:28.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.539+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f15ac0d0c20 con 0x7f15c4102fd0 2024-04-25T15:04:28.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.539+0000 7f15c1e00640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 0x7f15a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:04:28.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.540+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15ac09cf60 con 0x7f15c4102fd0 2024-04-25T15:04:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.740+0000 7f15c1e00640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 msgr2=0x7f15a80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/983233775 2024-04-25T15:04:28.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.740+0000 7f15c1e00640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 0x7f15a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:04:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:28.765+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f15c40758a0 con 0x7f15a80a3bf0 2024-04-25T15:04:29.063 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:28.759+0000 7f4a504a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:04:29.063 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.060+0000 7f4a504a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:04:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:29.085+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f15ac0995d0 con 0x7f15c4102fd0 2024-04-25T15:04:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:29.086+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 msgr2=0x7f15a80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:04:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:29.086+0000 7f15bb400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/983233775,v1:172.21.15.19:6801/983233775] conn(0x7f15a80a3bf0 0x7f15a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: Active manager daemon x restarted 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: Activating manager daemon x 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: mgrmap e21: x(active, starting, since 0.019317s), standbys: y, z 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:04:29.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:04:29.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:04:29.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-mon[28090]: Manager daemon x is now available 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: Active manager daemon x restarted 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: Activating manager daemon x 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: mgrmap e21: x(active, starting, since 0.019317s), standbys: y, z 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:04:29.380 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:04:29.381 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:04:29.382 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:04:29.383 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:04:29.383 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:04:29.383 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:29 smithi083 ceph-mon[32919]: Manager daemon x is now available 2024-04-25T15:04:29.385 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: Active manager daemon x restarted 2024-04-25T15:04:29.385 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: Activating manager daemon x 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: mgrmap e21: x(active, starting, since 0.019317s), standbys: y, z 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:04:29.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:04:29.387 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:04:29.388 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:29 smithi102 ceph-mon[33005]: Manager daemon x is now available 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.386+0000 7f49a4a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.386+0000 7f49a4a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.386+0000 7f49a4a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.386+0000 7f49a4a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.386+0000 7f49a4a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.387+0000 7f49a2200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.388+0000 7f49a2200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.388+0000 7f49a2200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.873 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.388+0000 7f49a2200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:29.873 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:04:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:04:29.388+0000 7f49a2200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:04:30.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.100+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mon.2 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f15ac0ce750 con 0x7f15c4102fd0 2024-04-25T15:04:30.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.101+0000 7f15bb400640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f15a8052820 0x7f15a8052c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:30.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.101+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f15c40758a0 con 0x7f15a8052820 2024-04-25T15:04:30.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.107+0000 7f15c1e00640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f15a8052820 0x7f15a8052c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.109+0000 7f15c1e00640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f15a8052820 0x7f15a8052c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005f40 tx=0x7f15b4009500 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:30.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.211+0000 7f15bb400640 1 -- 172.21.15.19:0/2844746300 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f15c40758a0 con 0x7f15a8052820 2024-04-25T15:04:30.214 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f15a8052820 msgr2=0x7f15a8052c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f15a8052820 0x7f15a8052c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005f40 tx=0x7f15b4009500 comp rx=0 tx=0).stop 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4102fd0 msgr2=0x7f15c419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 --2- 172.21.15.19:0/2844746300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c4102fd0 0x7f15c419a6e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e9f0 tx=0x7f15ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c2800640 1 -- 172.21.15.19:0/2844746300 reap_dead start 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 shutdown_connections 2024-04-25T15:04:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 >> 172.21.15.19:0/2844746300 conn(0x7f15c40fd350 msgr2=0x7f15c4074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 shutdown_connections 2024-04-25T15:04:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.214+0000 7f15c3200640 1 -- 172.21.15.19:0/2844746300 wait complete. 2024-04-25T15:04:30.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.378 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:30 smithi102 ceph-mon[33005]: mgrmap e22: x(active, since 1.0412s), standbys: y, z 2024-04-25T15:04:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 -- 172.21.15.19:0/4097022297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4100e60 msgr2=0x7f52e4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 --2- 172.21.15.19:0/4097022297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4100e60 0x7f52e4101280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009900 tx=0x7f52d402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 -- 172.21.15.19:0/4097022297 shutdown_connections 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 --2- 172.21.15.19:0/4097022297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4102a00 0x7f52e4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 --2- 172.21.15.19:0/4097022297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102040 0x7f52e41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 --2- 172.21.15.19:0/4097022297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4100e60 0x7f52e4101280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 -- 172.21.15.19:0/4097022297 >> 172.21.15.19:0/4097022297 conn(0x7f52e40fc420 msgr2=0x7f52e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 -- 172.21.15.19:0/4097022297 shutdown_connections 2024-04-25T15:04:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.423+0000 7f52e8c00640 1 -- 172.21.15.19:0/4097022297 wait complete. 2024-04-25T15:04:30.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 Processor -- start 2024-04-25T15:04:30.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 -- start start 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4100e60 0x7f52e419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102a00 0x7f52e41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410ca20 con 0x7f52e4100e60 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410c8a0 con 0x7f52e4102a00 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f52e410cba0 con 0x7f52e4102040 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4100e60 0x7f52e419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.425+0000 7f52e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:42342/0 (socket says 172.21.15.19:42342) 2024-04-25T15:04:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 -- 172.21.15.19:0/1069670527 learned_addr learned my addr 172.21.15.19:0/1069670527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e2800640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102a00 0x7f52e41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 -- 172.21.15.19:0/1069670527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102a00 msgr2=0x7f52e41a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102a00 0x7f52e41a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 -- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4100e60 msgr2=0x7f52e419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4100e60 0x7f52e419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 -- 172.21.15.19:0/1069670527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e41a1fb0 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.426+0000 7f52e1400640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009d50 tx=0x7f52cc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.427+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc003b70 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.427+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52cc003d10 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.427+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41a22a0 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.427+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0108a0 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.427+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41a2840 con 0x7f52e4102040 2024-04-25T15:04:30.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.428+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e41a2b00 con 0x7f52e4102040 2024-04-25T15:04:30.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.430+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f52cc004000 con 0x7f52e4102040 2024-04-25T15:04:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.432+0000 7f52daa00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 0x7f52c40a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.433+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f52cc0d0700 con 0x7f52e4102040 2024-04-25T15:04:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.433+0000 7f52e1e00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 0x7f52c40a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.434+0000 7f52e1e00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 0x7f52c40a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f52d4005ab0 tx=0x7f52d4005a20 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:30.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.436+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52cc09d2a0 con 0x7f52e4102040 2024-04-25T15:04:30.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:30.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:30 smithi083 ceph-mon[32919]: mgrmap e22: x(active, since 1.0412s), standbys: y, z 2024-04-25T15:04:30.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:30.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:04:30.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:04:30.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:30 smithi019 ceph-mon[28090]: mgrmap e22: x(active, since 1.0412s), standbys: y, z 2024-04-25T15:04:30.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.676+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f52e40fe5d0 con 0x7f52c40a3ae0 2024-04-25T15:04:30.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.808+0000 7f52daa00640 1 -- 172.21.15.19:0/1069670527 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f52e40fe5d0 con 0x7f52c40a3ae0 2024-04-25T15:04:30.809 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 msgr2=0x7f52c40a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 0x7f52c40a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f52d4005ab0 tx=0x7f52d4005a20 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 msgr2=0x7f52e419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009d50 tx=0x7f52cc00c6a0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 shutdown_connections 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52e4102a00 0x7f52e41a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f52c40a3ae0 0x7f52c40a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52e4102040 0x7f52e419a8d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 --2- 172.21.15.19:0/1069670527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e4100e60 0x7f52e419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.810+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 >> 172.21.15.19:0/1069670527 conn(0x7f52e40fc420 msgr2=0x7f52e40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.811+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 shutdown_connections 2024-04-25T15:04:30.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:30.811+0000 7f52e8c00640 1 -- 172.21.15.19:0/1069670527 wait complete. 2024-04-25T15:04:30.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 -- 172.21.15.19:0/1057185447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c103b60 msgr2=0x7fc19c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1057185447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c103b60 0x7fc19c10a3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc194009ca0 tx=0x7fc19402a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 -- 172.21.15.19:0/1057185447 shutdown_connections 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1057185447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c103b60 0x7fc19c10a3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1057185447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c1031a0 0x7fc19c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1057185447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c101fb0 0x7fc19c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.021+0000 7fc1a1000640 1 -- 172.21.15.19:0/1057185447 >> 172.21.15.19:0/1057185447 conn(0x7fc19c0fd540 msgr2=0x7fc19c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.022+0000 7fc1a1000640 1 -- 172.21.15.19:0/1057185447 shutdown_connections 2024-04-25T15:04:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.022+0000 7fc1a1000640 1 -- 172.21.15.19:0/1057185447 wait complete. 2024-04-25T15:04:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.023+0000 7fc1a1000640 1 Processor -- start 2024-04-25T15:04:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.023+0000 7fc1a1000640 1 -- start start 2024-04-25T15:04:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 0x7fc19c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c103b60 0x7fc19c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10db40 con 0x7fc19c103b60 2024-04-25T15:04:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10d9c0 con 0x7fc19c1031a0 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc1a1000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10dcc0 con 0x7fc19c101fb0 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc193e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc199e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 0x7fc19c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc193e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:33652/0 (socket says 172.21.15.19:33652) 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc193e00640 1 -- 172.21.15.19:0/1121649206 learned_addr learned my addr 172.21.15.19:0/1121649206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.024+0000 7fc19a800640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c103b60 0x7fc19c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 -- 172.21.15.19:0/1121649206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 msgr2=0x7fc19c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 0x7fc19c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 -- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c103b60 msgr2=0x7fc19c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c103b60 0x7fc19c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 -- 172.21.15.19:0/1121649206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19fca0 con 0x7fc19c1031a0 2024-04-25T15:04:31.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc199e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 0x7fc19c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc193e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cd80 con 0x7fc19c1031a0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1840108e0 con 0x7fc19c1031a0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184018980 con 0x7fc19c1031a0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.025+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19ff90 con 0x7fc19c1031a0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.026+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a0400 con 0x7fc19c1031a0 2024-04-25T15:04:31.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.027+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c109270 con 0x7fc19c1031a0 2024-04-25T15:04:31.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.029+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fc184010430 con 0x7fc19c1031a0 2024-04-25T15:04:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.030+0000 7fc192a00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 0x7fc1800a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.030+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc1840d06f0 con 0x7fc19c1031a0 2024-04-25T15:04:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.031+0000 7fc199e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 0x7fc1800a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.032+0000 7fc199e00640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 0x7fc1800a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc19c103000 tx=0x7fc18c005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.032+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc18409cc40 con 0x7fc19c1031a0 2024-04-25T15:04:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.317+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc19c062390 con 0x7fc1800a3ae0 2024-04-25T15:04:31.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.330+0000 7fc192a00640 1 -- 172.21.15.19:0/1121649206 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fc19c062390 con 0x7fc1800a3ae0 2024-04-25T15:04:31.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T15:04:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 msgr2=0x7fc1800a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 0x7fc1800a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc19c103000 tx=0x7fc18c005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 msgr2=0x7fc19c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 shutdown_connections 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19c103b60 0x7fc19c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc19c1031a0 0x7fc19c1985c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc1800a3ae0 0x7fc1800a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 --2- 172.21.15.19:0/1121649206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101fb0 0x7fc19c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.332+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 >> 172.21.15.19:0/1121649206 conn(0x7fc19c0fd540 msgr2=0x7fc19c106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.333+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 shutdown_connections 2024-04-25T15:04:31.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.333+0000 7fc1a1000640 1 -- 172.21.15.19:0/1121649206 wait complete. 2024-04-25T15:04:31.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T15:04:31.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: [25/Apr/2024:15:04:29] ENGINE Bus STARTING 2024-04-25T15:04:31.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: [25/Apr/2024:15:04:29] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T15:04:31.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: [25/Apr/2024:15:04:29] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T15:04:31.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: [25/Apr/2024:15:04:29] ENGINE Bus STARTED 2024-04-25T15:04:31.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:31.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:31.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:31 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: [25/Apr/2024:15:04:29] ENGINE Bus STARTING 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: [25/Apr/2024:15:04:29] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: [25/Apr/2024:15:04:29] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: [25/Apr/2024:15:04:29] ENGINE Bus STARTED 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.414 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.415 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:31 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 -- 172.21.15.19:0/3924651857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe010075f90 msgr2=0x7fe0100743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 --2- 172.21.15.19:0/3924651857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe010075f90 0x7fe0100743e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe000009890 tx=0x7fe00002a360 comp rx=0 tx=0).stop 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 -- 172.21.15.19:0/3924651857 shutdown_connections 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 --2- 172.21.15.19:0/3924651857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe01010e580 0x7fe010110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 --2- 172.21.15.19:0/3924651857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0100749b0 0x7fe010074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 --2- 172.21.15.19:0/3924651857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe010075f90 0x7fe0100743e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.541+0000 7fe018400640 1 -- 172.21.15.19:0/3924651857 >> 172.21.15.19:0/3924651857 conn(0x7fe0100fd540 msgr2=0x7fe0100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.542+0000 7fe018400640 1 -- 172.21.15.19:0/3924651857 shutdown_connections 2024-04-25T15:04:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.542+0000 7fe018400640 1 -- 172.21.15.19:0/3924651857 wait complete. 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.542+0000 7fe018400640 1 Processor -- start 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 -- start start 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe010075f90 0x7fe01019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101140b0 con 0x7fe010075f90 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe010113f30 con 0x7fe01010e580 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe018400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe010114230 con 0x7fe0100749b0 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe015000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe010075f90 0x7fe01019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe015a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe016400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe015a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:42390/0 (socket says 172.21.15.19:42390) 2024-04-25T15:04:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe016400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:33702/0 (socket says 172.21.15.19:33702) 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.543+0000 7fe015a00640 1 -- 172.21.15.19:0/764004359 learned_addr learned my addr 172.21.15.19:0/764004359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 -- 172.21.15.19:0/764004359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 msgr2=0x7fe0101a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 -- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe010075f90 msgr2=0x7fe01019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe010075f90 0x7fe01019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 -- 172.21.15.19:0/764004359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101a6360 con 0x7fe0100749b0 2024-04-25T15:04:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe016400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe015a00640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0000057c0 tx=0x7fe000032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000037040 con 0x7fe0100749b0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe000030520 con 0x7fe0100749b0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0000044e0 con 0x7fe0100749b0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.544+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101a65f0 con 0x7fe0100749b0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.545+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101a6a80 con 0x7fe0100749b0 2024-04-25T15:04:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.546+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe010104750 con 0x7fe0100749b0 2024-04-25T15:04:31.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.548+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fe000043020 con 0x7fe0100749b0 2024-04-25T15:04:31.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.549+0000 7fe00e000640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 0x7fdff80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:31.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.549+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe0000f0080 con 0x7fe0100749b0 2024-04-25T15:04:31.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.550+0000 7fe015000640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 0x7fdff80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:31.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.550+0000 7fe015000640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 0x7fdff80a6280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe01019fc50 tx=0x7fe004009500 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: [25/Apr/2024:15:04:29] ENGINE Bus STARTING 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: [25/Apr/2024:15:04:29] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: [25/Apr/2024:15:04:29] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: [25/Apr/2024:15:04:29] ENGINE Bus STARTED 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:31 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:31.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.556+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0000bc800 con 0x7fe0100749b0 2024-04-25T15:04:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.795+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe010062390 con 0x7fdff80a3dc0 2024-04-25T15:04:31.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.838+0000 7fe00e000640 1 -- 172.21.15.19:0/764004359 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe010062390 con 0x7fdff80a3dc0 2024-04-25T15:04:31.839 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T15:04:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 msgr2=0x7fdff80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 0x7fdff80a6280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe01019fc50 tx=0x7fe004009500 comp rx=0 tx=0).stop 2024-04-25T15:04:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 msgr2=0x7fe01019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:31.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0000057c0 tx=0x7fe000032d30 comp rx=0 tx=0).stop 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 shutdown_connections 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe01010e580 0x7fe0101a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe010075f90 0x7fe01019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fdff80a3dc0 0x7fdff80a6280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 --2- 172.21.15.19:0/764004359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0100749b0 0x7fe01019e740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 >> 172.21.15.19:0/764004359 conn(0x7fe0100fd540 msgr2=0x7fe010101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.841+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 shutdown_connections 2024-04-25T15:04:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:31.842+0000 7fe018400640 1 -- 172.21.15.19:0/764004359 wait complete. 2024-04-25T15:04:31.862 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T15:04:31.943 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T15:04:31.954 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T15:04:31.964 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:31.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T15:04:31.969 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T15:04:31.983 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T15:04:31.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:04:32.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='client.25202 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:32 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:32.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='client.25202 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T15:04:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:32 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='client.25202 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T15:04:32.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:32.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:32 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:33.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:04:33.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.625 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:33 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:33.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.793 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:33 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:33.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:33 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:34.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:34 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:04:34.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:34 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:04:34.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:34 smithi019 ceph-mon[28090]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:34.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:34 smithi019 ceph-mon[28090]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T15:04:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:34 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:04:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:34 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:04:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:34 smithi083 ceph-mon[32919]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:34.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:34 smithi083 ceph-mon[32919]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T15:04:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:34 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:04:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:34 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:04:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:34 smithi102 ceph-mon[33005]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:34.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:34 smithi102 ceph-mon[33005]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 -- 172.21.15.19:0/583848267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc1034b0 msgr2=0x7f18cc10f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 --2- 172.21.15.19:0/583848267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc1034b0 0x7f18cc10f920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18c400b0a0 tx=0x7f18c400b3b0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 -- 172.21.15.19:0/583848267 shutdown_connections 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 --2- 172.21.15.19:0/583848267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc1034b0 0x7f18cc10f920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 --2- 172.21.15.19:0/583848267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc107940 0x7f18cc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 --2- 172.21.15.19:0/583848267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc102af0 0x7f18cc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 -- 172.21.15.19:0/583848267 >> 172.21.15.19:0/583848267 conn(0x7f18cc079ba0 msgr2=0x7f18cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:34.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 -- 172.21.15.19:0/583848267 shutdown_connections 2024-04-25T15:04:34.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.857+0000 7f18d4600700 1 -- 172.21.15.19:0/583848267 wait complete. 2024-04-25T15:04:34.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.859+0000 7f18d4600700 1 Processor -- start 2024-04-25T15:04:34.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.859+0000 7f18d4600700 1 -- start start 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 0x7f18cc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc107940 0x7f18cc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112c90 con 0x7f18cc1034b0 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112b10 con 0x7f18cc107940 2024-04-25T15:04:34.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d4600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112e10 con 0x7f18cc102af0 2024-04-25T15:04:34.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 0x7f18cc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:34.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d1c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:34.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d1c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:45608/0 (socket says 172.21.15.19:45608) 2024-04-25T15:04:34.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 0x7f18cc199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59778/0 (socket says 172.21.15.19:59778) 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d1c00700 1 -- 172.21.15.19:0/2836981121 learned_addr learned my addr 172.21.15.19:0/2836981121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.860+0000 7f18d2600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc107940 0x7f18cc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 -- 172.21.15.19:0/2836981121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc107940 msgr2=0x7f18cc19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc107940 0x7f18cc19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 -- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 msgr2=0x7f18cc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 0x7f18cc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 -- 172.21.15.19:0/2836981121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc19e110 con 0x7f18cc102af0 2024-04-25T15:04:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.861+0000 7f18d1c00700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e9a0 tx=0x7f18c000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:34.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.862+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c000ca10 con 0x7f18cc102af0 2024-04-25T15:04:34.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.862+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc19e400 con 0x7f18cc102af0 2024-04-25T15:04:34.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.862+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c000cbb0 con 0x7f18cc102af0 2024-04-25T15:04:34.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.862+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0018680 con 0x7f18cc102af0 2024-04-25T15:04:34.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.863+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a5c80 con 0x7f18cc102af0 2024-04-25T15:04:34.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.864+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc069730 con 0x7f18cc102af0 2024-04-25T15:04:34.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.867+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f18c0010430 con 0x7f18cc102af0 2024-04-25T15:04:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.868+0000 7f18ca000700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 0x7f18b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.868+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f18c00d0a80 con 0x7f18cc102af0 2024-04-25T15:04:34.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.869+0000 7f18d1200700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 0x7f18b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:34.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.869+0000 7f18d1200700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 0x7f18b40a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009680 tx=0x7f18b8009260 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:34.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:34.872+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18c00d2e20 con 0x7f18cc102af0 2024-04-25T15:04:35.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.110+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18cc19a460 con 0x7f18b40a3e40 2024-04-25T15:04:35.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.124+0000 7f18ca000700 1 -- 172.21.15.19:0/2836981121 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257688 (secure 0 0 0) 0x7f18cc19a460 con 0x7f18b40a3e40 2024-04-25T15:04:35.130 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.128+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 msgr2=0x7f18b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.128+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 0x7f18b40a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009680 tx=0x7f18b8009260 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 msgr2=0x7f18cc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e9a0 tx=0x7f18c000ecb0 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 shutdown_connections 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f18b40a3e40 0x7f18b40a6300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18cc107940 0x7f18cc19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18cc102af0 0x7f18cc199100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 --2- 172.21.15.19:0/2836981121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18cc1034b0 0x7f18cc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:35.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 >> 172.21.15.19:0/2836981121 conn(0x7f18cc079ba0 msgr2=0x7f18cc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:35.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 shutdown_connections 2024-04-25T15:04:35.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:35.129+0000 7f18d4600700 1 -- 172.21.15.19:0/2836981121 wait complete. 2024-04-25T15:04:35.134 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:04:35.998 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T15:04:35.104340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183462,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":539,"num_read_kb":422,"num_write":387,"num_write_kb":4346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389492,"kb_used_data":15152,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552332,"statfs":{"total":1151940427776,"available":1151541587968,"internally_reserved":0,"allocated":15515648,"data_stored":9286447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.980339"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641948+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:04:29.641948+0000","last_peered":"2024-04-25T15:04:29.641948+0000","last_clean":"2024-04-25T15:04:29.641948+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:04:29.641948+0000","last_undegraded":"2024-04-25T15:04:29.641948+0000","last_fullsized":"2024-04-25T15:04:29.641948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507665+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:04:29.507665+0000","last_peered":"2024-04-25T15:04:29.507665+0000","last_clean":"2024-04-25T15:04:29.507665+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:04:29.507665+0000","last_undegraded":"2024-04-25T15:04:29.507665+0000","last_fullsized":"2024-04-25T15:04:29.507665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507262+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:04:29.507262+0000","last_peered":"2024-04-25T15:04:29.507262+0000","last_clean":"2024-04-25T15:04:29.507262+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:04:29.507262+0000","last_undegraded":"2024-04-25T15:04:29.507262+0000","last_fullsized":"2024-04-25T15:04:29.507262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083735+0000","last_change":"2024-04-25T15:02:57.474358+0000","last_active":"2024-04-25T15:04:29.083735+0000","last_peered":"2024-04-25T15:04:29.083735+0000","last_clean":"2024-04-25T15:04:29.083735+0000","last_became_active":"2024-04-25T15:02:57.473924+0000","last_became_peered":"2024-04-25T15:02:57.473924+0000","last_unstale":"2024-04-25T15:04:29.083735+0000","last_undegraded":"2024-04-25T15:04:29.083735+0000","last_fullsized":"2024-04-25T15:04:29.083735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.681108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205528+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:04:29.205528+0000","last_peered":"2024-04-25T15:04:29.205528+0000","last_clean":"2024-04-25T15:04:29.205528+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:04:29.205528+0000","last_undegraded":"2024-04-25T15:04:29.205528+0000","last_fullsized":"2024-04-25T15:04:29.205528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217099+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:04:29.217099+0000","last_peered":"2024-04-25T15:04:29.217099+0000","last_clean":"2024-04-25T15:04:29.217099+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:04:29.217099+0000","last_undegraded":"2024-04-25T15:04:29.217099+0000","last_fullsized":"2024-04-25T15:04:29.217099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354158+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:04:29.354158+0000","last_peered":"2024-04-25T15:04:29.354158+0000","last_clean":"2024-04-25T15:04:29.354158+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:04:29.354158+0000","last_undegraded":"2024-04-25T15:04:29.354158+0000","last_fullsized":"2024-04-25T15:04:29.354158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205987+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:04:29.205987+0000","last_peered":"2024-04-25T15:04:29.205987+0000","last_clean":"2024-04-25T15:04:29.205987+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:04:29.205987+0000","last_undegraded":"2024-04-25T15:04:29.205987+0000","last_fullsized":"2024-04-25T15:04:29.205987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325092+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:04:29.325092+0000","last_peered":"2024-04-25T15:04:29.325092+0000","last_clean":"2024-04-25T15:04:29.325092+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:04:29.325092+0000","last_undegraded":"2024-04-25T15:04:29.325092+0000","last_fullsized":"2024-04-25T15:04:29.325092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217524+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:04:29.217524+0000","last_peered":"2024-04-25T15:04:29.217524+0000","last_clean":"2024-04-25T15:04:29.217524+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:04:29.217524+0000","last_undegraded":"2024-04-25T15:04:29.217524+0000","last_fullsized":"2024-04-25T15:04:29.217524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354547+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:04:29.354547+0000","last_peered":"2024-04-25T15:04:29.354547+0000","last_clean":"2024-04-25T15:04:29.354547+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:04:29.354547+0000","last_undegraded":"2024-04-25T15:04:29.354547+0000","last_fullsized":"2024-04-25T15:04:29.354547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.078119+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:04:29.078119+0000","last_peered":"2024-04-25T15:04:29.078119+0000","last_clean":"2024-04-25T15:04:29.078119+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:04:29.078119+0000","last_undegraded":"2024-04-25T15:04:29.078119+0000","last_fullsized":"2024-04-25T15:04:29.078119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642134+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:04:29.642134+0000","last_peered":"2024-04-25T15:04:29.642134+0000","last_clean":"2024-04-25T15:04:29.642134+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:04:29.642134+0000","last_undegraded":"2024-04-25T15:04:29.642134+0000","last_fullsized":"2024-04-25T15:04:29.642134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507345+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:04:29.507345+0000","last_peered":"2024-04-25T15:04:29.507345+0000","last_clean":"2024-04-25T15:04:29.507345+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:04:29.507345+0000","last_undegraded":"2024-04-25T15:04:29.507345+0000","last_fullsized":"2024-04-25T15:04:29.507345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507098+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:04:29.507098+0000","last_peered":"2024-04-25T15:04:29.507098+0000","last_clean":"2024-04-25T15:04:29.507098+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:04:29.507098+0000","last_undegraded":"2024-04-25T15:04:29.507098+0000","last_fullsized":"2024-04-25T15:04:29.507098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324603+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:04:29.324603+0000","last_peered":"2024-04-25T15:04:29.324603+0000","last_clean":"2024-04-25T15:04:29.324603+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:04:29.324603+0000","last_undegraded":"2024-04-25T15:04:29.324603+0000","last_fullsized":"2024-04-25T15:04:29.324603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642011+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:04:29.642011+0000","last_peered":"2024-04-25T15:04:29.642011+0000","last_clean":"2024-04-25T15:04:29.642011+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:04:29.642011+0000","last_undegraded":"2024-04-25T15:04:29.642011+0000","last_fullsized":"2024-04-25T15:04:29.642011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083738+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:04:29.083738+0000","last_peered":"2024-04-25T15:04:29.083738+0000","last_clean":"2024-04-25T15:04:29.083738+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:04:29.083738+0000","last_undegraded":"2024-04-25T15:04:29.083738+0000","last_fullsized":"2024-04-25T15:04:29.083738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079371+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:04:29.079371+0000","last_peered":"2024-04-25T15:04:29.079371+0000","last_clean":"2024-04-25T15:04:29.079371+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:04:29.079371+0000","last_undegraded":"2024-04-25T15:04:29.079371+0000","last_fullsized":"2024-04-25T15:04:29.079371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.085077+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:04:29.085077+0000","last_peered":"2024-04-25T15:04:29.085077+0000","last_clean":"2024-04-25T15:04:29.085077+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:04:29.085077+0000","last_undegraded":"2024-04-25T15:04:29.085077+0000","last_fullsized":"2024-04-25T15:04:29.085077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205959+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:04:29.205959+0000","last_peered":"2024-04-25T15:04:29.205959+0000","last_clean":"2024-04-25T15:04:29.205959+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:04:29.205959+0000","last_undegraded":"2024-04-25T15:04:29.205959+0000","last_fullsized":"2024-04-25T15:04:29.205959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206088+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:04:29.206088+0000","last_peered":"2024-04-25T15:04:29.206088+0000","last_clean":"2024-04-25T15:04:29.206088+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:04:29.206088+0000","last_undegraded":"2024-04-25T15:04:29.206088+0000","last_fullsized":"2024-04-25T15:04:29.206088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507821+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:04:29.507821+0000","last_peered":"2024-04-25T15:04:29.507821+0000","last_clean":"2024-04-25T15:04:29.507821+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:04:29.507821+0000","last_undegraded":"2024-04-25T15:04:29.507821+0000","last_fullsized":"2024-04-25T15:04:29.507821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205973+0000","last_change":"2024-04-25T15:02:57.476182+0000","last_active":"2024-04-25T15:04:29.205973+0000","last_peered":"2024-04-25T15:04:29.205973+0000","last_clean":"2024-04-25T15:04:29.205973+0000","last_became_active":"2024-04-25T15:02:57.472985+0000","last_became_peered":"2024-04-25T15:02:57.472985+0000","last_unstale":"2024-04-25T15:04:29.205973+0000","last_undegraded":"2024-04-25T15:04:29.205973+0000","last_fullsized":"2024-04-25T15:04:29.205973+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.669296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325105+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:04:29.325105+0000","last_peered":"2024-04-25T15:04:29.325105+0000","last_clean":"2024-04-25T15:04:29.325105+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:04:29.325105+0000","last_undegraded":"2024-04-25T15:04:29.325105+0000","last_fullsized":"2024-04-25T15:04:29.325105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205852+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:04:29.205852+0000","last_peered":"2024-04-25T15:04:29.205852+0000","last_clean":"2024-04-25T15:04:29.205852+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:04:29.205852+0000","last_undegraded":"2024-04-25T15:04:29.205852+0000","last_fullsized":"2024-04-25T15:04:29.205852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213902+0000","last_change":"2024-04-25T15:02:57.471043+0000","last_active":"2024-04-25T15:04:29.213902+0000","last_peered":"2024-04-25T15:04:29.213902+0000","last_clean":"2024-04-25T15:04:29.213902+0000","last_became_active":"2024-04-25T15:02:57.470670+0000","last_became_peered":"2024-04-25T15:02:57.470670+0000","last_unstale":"2024-04-25T15:04:29.213902+0000","last_undegraded":"2024-04-25T15:04:29.213902+0000","last_fullsized":"2024-04-25T15:04:29.213902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.243201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213843+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:04:29.213843+0000","last_peered":"2024-04-25T15:04:29.213843+0000","last_clean":"2024-04-25T15:04:29.213843+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:04:29.213843+0000","last_undegraded":"2024-04-25T15:04:29.213843+0000","last_fullsized":"2024-04-25T15:04:29.213843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205893+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:04:29.205893+0000","last_peered":"2024-04-25T15:04:29.205893+0000","last_clean":"2024-04-25T15:04:29.205893+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:04:29.205893+0000","last_undegraded":"2024-04-25T15:04:29.205893+0000","last_fullsized":"2024-04-25T15:04:29.205893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217119+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:04:29.217119+0000","last_peered":"2024-04-25T15:04:29.217119+0000","last_clean":"2024-04-25T15:04:29.217119+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:04:29.217119+0000","last_undegraded":"2024-04-25T15:04:29.217119+0000","last_fullsized":"2024-04-25T15:04:29.217119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213785+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:04:29.213785+0000","last_peered":"2024-04-25T15:04:29.213785+0000","last_clean":"2024-04-25T15:04:29.213785+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:04:29.213785+0000","last_undegraded":"2024-04-25T15:04:29.213785+0000","last_fullsized":"2024-04-25T15:04:29.213785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642227+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:04:29.642227+0000","last_peered":"2024-04-25T15:04:29.642227+0000","last_clean":"2024-04-25T15:04:29.642227+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:04:29.642227+0000","last_undegraded":"2024-04-25T15:04:29.642227+0000","last_fullsized":"2024-04-25T15:04:29.642227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205603+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:04:29.205603+0000","last_peered":"2024-04-25T15:04:29.205603+0000","last_clean":"2024-04-25T15:04:29.205603+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:04:29.205603+0000","last_undegraded":"2024-04-25T15:04:29.205603+0000","last_fullsized":"2024-04-25T15:04:29.205603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213605+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:04:29.213605+0000","last_peered":"2024-04-25T15:04:29.213605+0000","last_clean":"2024-04-25T15:04:29.213605+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:04:29.213605+0000","last_undegraded":"2024-04-25T15:04:29.213605+0000","last_fullsized":"2024-04-25T15:04:29.213605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083091+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:04:29.083091+0000","last_peered":"2024-04-25T15:04:29.083091+0000","last_clean":"2024-04-25T15:04:29.083091+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:04:29.083091+0000","last_undegraded":"2024-04-25T15:04:29.083091+0000","last_fullsized":"2024-04-25T15:04:29.083091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217382+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:04:29.217382+0000","last_peered":"2024-04-25T15:04:29.217382+0000","last_clean":"2024-04-25T15:04:29.217382+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:04:29.217382+0000","last_undegraded":"2024-04-25T15:04:29.217382+0000","last_fullsized":"2024-04-25T15:04:29.217382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325069+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:04:29.325069+0000","last_peered":"2024-04-25T15:04:29.325069+0000","last_clean":"2024-04-25T15:04:29.325069+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:04:29.325069+0000","last_undegraded":"2024-04-25T15:04:29.325069+0000","last_fullsized":"2024-04-25T15:04:29.325069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354452+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:04:29.354452+0000","last_peered":"2024-04-25T15:04:29.354452+0000","last_clean":"2024-04-25T15:04:29.354452+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:04:29.354452+0000","last_undegraded":"2024-04-25T15:04:29.354452+0000","last_fullsized":"2024-04-25T15:04:29.354452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507878+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:04:29.507878+0000","last_peered":"2024-04-25T15:04:29.507878+0000","last_clean":"2024-04-25T15:04:29.507878+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:04:29.507878+0000","last_undegraded":"2024-04-25T15:04:29.507878+0000","last_fullsized":"2024-04-25T15:04:29.507878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.214112+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:04:29.214112+0000","last_peered":"2024-04-25T15:04:29.214112+0000","last_clean":"2024-04-25T15:04:29.214112+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:04:29.214112+0000","last_undegraded":"2024-04-25T15:04:29.214112+0000","last_fullsized":"2024-04-25T15:04:29.214112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325015+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:04:29.325015+0000","last_peered":"2024-04-25T15:04:29.325015+0000","last_clean":"2024-04-25T15:04:29.325015+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:04:29.325015+0000","last_undegraded":"2024-04-25T15:04:29.325015+0000","last_fullsized":"2024-04-25T15:04:29.325015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":361,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.620031+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:04:29.620031+0000","last_peered":"2024-04-25T15:04:29.620031+0000","last_clean":"2024-04-25T15:04:29.620031+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:04:29.620031+0000","last_undegraded":"2024-04-25T15:04:29.620031+0000","last_fullsized":"2024-04-25T15:04:29.620031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67246,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":304,"num_read_kb":218,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079551+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:04:29.079551+0000","last_peered":"2024-04-25T15:04:29.079551+0000","last_clean":"2024-04-25T15:04:29.079551+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:04:29.079551+0000","last_undegraded":"2024-04-25T15:04:29.079551+0000","last_fullsized":"2024-04-25T15:04:29.079551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641849+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:04:29.641849+0000","last_peered":"2024-04-25T15:04:29.641849+0000","last_clean":"2024-04-25T15:04:29.641849+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:04:29.641849+0000","last_undegraded":"2024-04-25T15:04:29.641849+0000","last_fullsized":"2024-04-25T15:04:29.641849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213964+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:04:29.213964+0000","last_peered":"2024-04-25T15:04:29.213964+0000","last_clean":"2024-04-25T15:04:29.213964+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:04:29.213964+0000","last_undegraded":"2024-04-25T15:04:29.213964+0000","last_fullsized":"2024-04-25T15:04:29.213964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354351+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:04:29.354351+0000","last_peered":"2024-04-25T15:04:29.354351+0000","last_clean":"2024-04-25T15:04:29.354351+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:04:29.354351+0000","last_undegraded":"2024-04-25T15:04:29.354351+0000","last_fullsized":"2024-04-25T15:04:29.354351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077852+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:04:29.077852+0000","last_peered":"2024-04-25T15:04:29.077852+0000","last_clean":"2024-04-25T15:04:29.077852+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:04:29.077852+0000","last_undegraded":"2024-04-25T15:04:29.077852+0000","last_fullsized":"2024-04-25T15:04:29.077852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354095+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:04:29.354095+0000","last_peered":"2024-04-25T15:04:29.354095+0000","last_clean":"2024-04-25T15:04:29.354095+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:04:29.354095+0000","last_undegraded":"2024-04-25T15:04:29.354095+0000","last_fullsized":"2024-04-25T15:04:29.354095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205561+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:04:29.205561+0000","last_peered":"2024-04-25T15:04:29.205561+0000","last_clean":"2024-04-25T15:04:29.205561+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:04:29.205561+0000","last_undegraded":"2024-04-25T15:04:29.205561+0000","last_fullsized":"2024-04-25T15:04:29.205561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213715+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:04:29.213715+0000","last_peered":"2024-04-25T15:04:29.213715+0000","last_clean":"2024-04-25T15:04:29.213715+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:04:29.213715+0000","last_undegraded":"2024-04-25T15:04:29.213715+0000","last_fullsized":"2024-04-25T15:04:29.213715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641863+0000","last_change":"2024-04-25T15:02:57.473783+0000","last_active":"2024-04-25T15:04:29.641863+0000","last_peered":"2024-04-25T15:04:29.641863+0000","last_clean":"2024-04-25T15:04:29.641863+0000","last_became_active":"2024-04-25T15:02:57.468466+0000","last_became_peered":"2024-04-25T15:02:57.468466+0000","last_unstale":"2024-04-25T15:04:29.641863+0000","last_undegraded":"2024-04-25T15:04:29.641863+0000","last_fullsized":"2024-04-25T15:04:29.641863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:41.182777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641910+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:04:29.641910+0000","last_peered":"2024-04-25T15:04:29.641910+0000","last_clean":"2024-04-25T15:04:29.641910+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:04:29.641910+0000","last_undegraded":"2024-04-25T15:04:29.641910+0000","last_fullsized":"2024-04-25T15:04:29.641910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083393+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:04:29.083393+0000","last_peered":"2024-04-25T15:04:29.083393+0000","last_clean":"2024-04-25T15:04:29.083393+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:04:29.083393+0000","last_undegraded":"2024-04-25T15:04:29.083393+0000","last_fullsized":"2024-04-25T15:04:29.083393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217387+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:04:29.217387+0000","last_peered":"2024-04-25T15:04:29.217387+0000","last_clean":"2024-04-25T15:04:29.217387+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:04:29.217387+0000","last_undegraded":"2024-04-25T15:04:29.217387+0000","last_fullsized":"2024-04-25T15:04:29.217387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354189+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:04:29.354189+0000","last_peered":"2024-04-25T15:04:29.354189+0000","last_clean":"2024-04-25T15:04:29.354189+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:04:29.354189+0000","last_undegraded":"2024-04-25T15:04:29.354189+0000","last_fullsized":"2024-04-25T15:04:29.354189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217231+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:04:29.217231+0000","last_peered":"2024-04-25T15:04:29.217231+0000","last_clean":"2024-04-25T15:04:29.217231+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:04:29.217231+0000","last_undegraded":"2024-04-25T15:04:29.217231+0000","last_fullsized":"2024-04-25T15:04:29.217231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213699+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:04:29.213699+0000","last_peered":"2024-04-25T15:04:29.213699+0000","last_clean":"2024-04-25T15:04:29.213699+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:04:29.213699+0000","last_undegraded":"2024-04-25T15:04:29.213699+0000","last_fullsized":"2024-04-25T15:04:29.213699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641750+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:04:29.641750+0000","last_peered":"2024-04-25T15:04:29.641750+0000","last_clean":"2024-04-25T15:04:29.641750+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:04:29.641750+0000","last_undegraded":"2024-04-25T15:04:29.641750+0000","last_fullsized":"2024-04-25T15:04:29.641750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205739+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:04:29.205739+0000","last_peered":"2024-04-25T15:04:29.205739+0000","last_clean":"2024-04-25T15:04:29.205739+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:04:29.205739+0000","last_undegraded":"2024-04-25T15:04:29.205739+0000","last_fullsized":"2024-04-25T15:04:29.205739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077898+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:04:29.077898+0000","last_peered":"2024-04-25T15:04:29.077898+0000","last_clean":"2024-04-25T15:04:29.077898+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:04:29.077898+0000","last_undegraded":"2024-04-25T15:04:29.077898+0000","last_fullsized":"2024-04-25T15:04:29.077898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354130+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:04:29.354130+0000","last_peered":"2024-04-25T15:04:29.354130+0000","last_clean":"2024-04-25T15:04:29.354130+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:04:29.354130+0000","last_undegraded":"2024-04-25T15:04:29.354130+0000","last_fullsized":"2024-04-25T15:04:29.354130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641602+0000","last_change":"2024-04-25T15:02:57.474003+0000","last_active":"2024-04-25T15:04:29.641602+0000","last_peered":"2024-04-25T15:04:29.641602+0000","last_clean":"2024-04-25T15:04:29.641602+0000","last_became_active":"2024-04-25T15:02:57.473443+0000","last_became_peered":"2024-04-25T15:02:57.473443+0000","last_unstale":"2024-04-25T15:04:29.641602+0000","last_undegraded":"2024-04-25T15:04:29.641602+0000","last_fullsized":"2024-04-25T15:04:29.641602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:30.170675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354286+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:04:29.354286+0000","last_peered":"2024-04-25T15:04:29.354286+0000","last_clean":"2024-04-25T15:04:29.354286+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:04:29.354286+0000","last_undegraded":"2024-04-25T15:04:29.354286+0000","last_fullsized":"2024-04-25T15:04:29.354286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641817+0000","last_change":"2024-04-25T15:02:57.529669+0000","last_active":"2024-04-25T15:04:29.641817+0000","last_peered":"2024-04-25T15:04:29.641817+0000","last_clean":"2024-04-25T15:04:29.641817+0000","last_became_active":"2024-04-25T15:02:57.529390+0000","last_became_peered":"2024-04-25T15:02:57.529390+0000","last_unstale":"2024-04-25T15:04:29.641817+0000","last_undegraded":"2024-04-25T15:04:29.641817+0000","last_fullsized":"2024-04-25T15:04:29.641817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:36.255304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642335+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:04:29.642335+0000","last_peered":"2024-04-25T15:04:29.642335+0000","last_clean":"2024-04-25T15:04:29.642335+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:04:29.642335+0000","last_undegraded":"2024-04-25T15:04:29.642335+0000","last_fullsized":"2024-04-25T15:04:29.642335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'46","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:30.073050+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:04:30.073050+0000","last_peered":"2024-04-25T15:04:30.073050+0000","last_clean":"2024-04-25T15:04:30.073050+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:04:30.073050+0000","last_undegraded":"2024-04-25T15:04:30.073050+0000","last_fullsized":"2024-04-25T15:04:30.073050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":83,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206049+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:04:29.206049+0000","last_peered":"2024-04-25T15:04:29.206049+0000","last_clean":"2024-04-25T15:04:29.206049+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:04:29.206049+0000","last_undegraded":"2024-04-25T15:04:29.206049+0000","last_fullsized":"2024-04-25T15:04:29.206049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077925+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:04:29.077925+0000","last_peered":"2024-04-25T15:04:29.077925+0000","last_clean":"2024-04-25T15:04:29.077925+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:04:29.077925+0000","last_undegraded":"2024-04-25T15:04:29.077925+0000","last_fullsized":"2024-04-25T15:04:29.077925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507811+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:04:29.507811+0000","last_peered":"2024-04-25T15:04:29.507811+0000","last_clean":"2024-04-25T15:04:29.507811+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:04:29.507811+0000","last_undegraded":"2024-04-25T15:04:29.507811+0000","last_fullsized":"2024-04-25T15:04:29.507811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324508+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:04:29.324508+0000","last_peered":"2024-04-25T15:04:29.324508+0000","last_clean":"2024-04-25T15:04:29.324508+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:04:29.324508+0000","last_undegraded":"2024-04-25T15:04:29.324508+0000","last_fullsized":"2024-04-25T15:04:29.324508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205471+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:04:29.205471+0000","last_peered":"2024-04-25T15:04:29.205471+0000","last_clean":"2024-04-25T15:04:29.205471+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:04:29.205471+0000","last_undegraded":"2024-04-25T15:04:29.205471+0000","last_fullsized":"2024-04-25T15:04:29.205471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206025+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:04:29.206025+0000","last_peered":"2024-04-25T15:04:29.206025+0000","last_clean":"2024-04-25T15:04:29.206025+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:04:29.206025+0000","last_undegraded":"2024-04-25T15:04:29.206025+0000","last_fullsized":"2024-04-25T15:04:29.206025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641669+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:04:29.641669+0000","last_peered":"2024-04-25T15:04:29.641669+0000","last_clean":"2024-04-25T15:04:29.641669+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:04:29.641669+0000","last_undegraded":"2024-04-25T15:04:29.641669+0000","last_fullsized":"2024-04-25T15:04:29.641669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079203+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:04:29.079203+0000","last_peered":"2024-04-25T15:04:29.079203+0000","last_clean":"2024-04-25T15:04:29.079203+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:04:29.079203+0000","last_undegraded":"2024-04-25T15:04:29.079203+0000","last_fullsized":"2024-04-25T15:04:29.079203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205734+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:04:29.205734+0000","last_peered":"2024-04-25T15:04:29.205734+0000","last_clean":"2024-04-25T15:04:29.205734+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:04:29.205734+0000","last_undegraded":"2024-04-25T15:04:29.205734+0000","last_fullsized":"2024-04-25T15:04:29.205734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205592+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:04:29.205592+0000","last_peered":"2024-04-25T15:04:29.205592+0000","last_clean":"2024-04-25T15:04:29.205592+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:04:29.205592+0000","last_undegraded":"2024-04-25T15:04:29.205592+0000","last_fullsized":"2024-04-25T15:04:29.205592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354285+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:04:29.354285+0000","last_peered":"2024-04-25T15:04:29.354285+0000","last_clean":"2024-04-25T15:04:29.354285+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:04:29.354285+0000","last_undegraded":"2024-04-25T15:04:29.354285+0000","last_fullsized":"2024-04-25T15:04:29.354285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324613+0000","last_change":"2024-04-25T15:02:57.529816+0000","last_active":"2024-04-25T15:04:29.324613+0000","last_peered":"2024-04-25T15:04:29.324613+0000","last_clean":"2024-04-25T15:04:29.324613+0000","last_became_active":"2024-04-25T15:02:57.529442+0000","last_became_peered":"2024-04-25T15:02:57.529442+0000","last_unstale":"2024-04-25T15:04:29.324613+0000","last_undegraded":"2024-04-25T15:04:29.324613+0000","last_fullsized":"2024-04-25T15:04:29.324613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.143614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354562+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:04:29.354562+0000","last_peered":"2024-04-25T15:04:29.354562+0000","last_clean":"2024-04-25T15:04:29.354562+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:04:29.354562+0000","last_undegraded":"2024-04-25T15:04:29.354562+0000","last_fullsized":"2024-04-25T15:04:29.354562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.342688+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:04:29.342688+0000","last_peered":"2024-04-25T15:04:29.342688+0000","last_clean":"2024-04-25T15:04:29.342688+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:04:29.342688+0000","last_undegraded":"2024-04-25T15:04:29.342688+0000","last_fullsized":"2024-04-25T15:04:29.342688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354311+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:04:29.354311+0000","last_peered":"2024-04-25T15:04:29.354311+0000","last_clean":"2024-04-25T15:04:29.354311+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:04:29.354311+0000","last_undegraded":"2024-04-25T15:04:29.354311+0000","last_fullsized":"2024-04-25T15:04:29.354311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642550+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:04:29.642550+0000","last_peered":"2024-04-25T15:04:29.642550+0000","last_clean":"2024-04-25T15:04:29.642550+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:04:29.642550+0000","last_undegraded":"2024-04-25T15:04:29.642550+0000","last_fullsized":"2024-04-25T15:04:29.642550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354512+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:04:29.354512+0000","last_peered":"2024-04-25T15:04:29.354512+0000","last_clean":"2024-04-25T15:04:29.354512+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:04:29.354512+0000","last_undegraded":"2024-04-25T15:04:29.354512+0000","last_fullsized":"2024-04-25T15:04:29.354512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641721+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:04:29.641721+0000","last_peered":"2024-04-25T15:04:29.641721+0000","last_clean":"2024-04-25T15:04:29.641721+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:04:29.641721+0000","last_undegraded":"2024-04-25T15:04:29.641721+0000","last_fullsized":"2024-04-25T15:04:29.641721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079375+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:04:29.079375+0000","last_peered":"2024-04-25T15:04:29.079375+0000","last_clean":"2024-04-25T15:04:29.079375+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:04:29.079375+0000","last_undegraded":"2024-04-25T15:04:29.079375+0000","last_fullsized":"2024-04-25T15:04:29.079375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079220+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:04:29.079220+0000","last_peered":"2024-04-25T15:04:29.079220+0000","last_clean":"2024-04-25T15:04:29.079220+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:04:29.079220+0000","last_undegraded":"2024-04-25T15:04:29.079220+0000","last_fullsized":"2024-04-25T15:04:29.079220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507354+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:04:29.507354+0000","last_peered":"2024-04-25T15:04:29.507354+0000","last_clean":"2024-04-25T15:04:29.507354+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:04:29.507354+0000","last_undegraded":"2024-04-25T15:04:29.507354+0000","last_fullsized":"2024-04-25T15:04:29.507354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642445+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:04:29.642445+0000","last_peered":"2024-04-25T15:04:29.642445+0000","last_clean":"2024-04-25T15:04:29.642445+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:04:29.642445+0000","last_undegraded":"2024-04-25T15:04:29.642445+0000","last_fullsized":"2024-04-25T15:04:29.642445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641969+0000","last_change":"2024-04-25T15:02:57.470460+0000","last_active":"2024-04-25T15:04:29.641969+0000","last_peered":"2024-04-25T15:04:29.641969+0000","last_clean":"2024-04-25T15:04:29.641969+0000","last_became_active":"2024-04-25T15:02:57.470013+0000","last_became_peered":"2024-04-25T15:02:57.470013+0000","last_unstale":"2024-04-25T15:04:29.641969+0000","last_undegraded":"2024-04-25T15:04:29.641969+0000","last_fullsized":"2024-04-25T15:04:29.641969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:07.351115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079804+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:04:29.079804+0000","last_peered":"2024-04-25T15:04:29.079804+0000","last_clean":"2024-04-25T15:04:29.079804+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:04:29.079804+0000","last_undegraded":"2024-04-25T15:04:29.079804+0000","last_fullsized":"2024-04-25T15:04:29.079804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213572+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:04:29.213572+0000","last_peered":"2024-04-25T15:04:29.213572+0000","last_clean":"2024-04-25T15:04:29.213572+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:04:29.213572+0000","last_undegraded":"2024-04-25T15:04:29.213572+0000","last_fullsized":"2024-04-25T15:04:29.213572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205767+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:04:29.205767+0000","last_peered":"2024-04-25T15:04:29.205767+0000","last_clean":"2024-04-25T15:04:29.205767+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:04:29.205767+0000","last_undegraded":"2024-04-25T15:04:29.205767+0000","last_fullsized":"2024-04-25T15:04:29.205767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324892+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:04:29.324892+0000","last_peered":"2024-04-25T15:04:29.324892+0000","last_clean":"2024-04-25T15:04:29.324892+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:04:29.324892+0000","last_undegraded":"2024-04-25T15:04:29.324892+0000","last_fullsized":"2024-04-25T15:04:29.324892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642103+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:04:29.642103+0000","last_peered":"2024-04-25T15:04:29.642103+0000","last_clean":"2024-04-25T15:04:29.642103+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:04:29.642103+0000","last_undegraded":"2024-04-25T15:04:29.642103+0000","last_fullsized":"2024-04-25T15:04:29.642103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641698+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:04:29.641698+0000","last_peered":"2024-04-25T15:04:29.641698+0000","last_clean":"2024-04-25T15:04:29.641698+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:04:29.641698+0000","last_undegraded":"2024-04-25T15:04:29.641698+0000","last_fullsized":"2024-04-25T15:04:29.641698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":592860,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":403,"num_read_kb":303,"num_write":204,"num_write_kb":1406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1778580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285610,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547249,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1268274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972413,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1589248,"data_stored":1069816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":545995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":974848,"data_stored":469912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":546029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1531904,"data_stored":1003903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888147,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1732608,"data_stored":1201016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:04:36.003 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:04:36.290 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:35 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:35 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:35 smithi083 ceph-mon[32919]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:36.291 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:35 smithi083 ceph-mon[32919]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:36.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:35 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:35 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:35 smithi102 ceph-mon[33005]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:36.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:35 smithi102 ceph-mon[33005]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:36.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:35 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:35 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:36.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:35 smithi019 ceph-mon[28090]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:04:36.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:35 smithi019 ceph-mon[28090]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:36.427 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:38.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T15:04:38.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T15:04:38.623 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:38 smithi019 ceph-mon[28090]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T15:04:38.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:38 smithi083 ceph-mon[32919]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T15:04:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 -- 172.21.15.19:0/1486979827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 msgr2=0x7fab3006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 --2- 172.21.15.19:0/1486979827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3006abb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0098f0 tx=0x7fab1c009c00 comp rx=0 tx=0).stop 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 -- 172.21.15.19:0/1486979827 shutdown_connections 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 --2- 172.21.15.19:0/1486979827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab3006b0f0 0x7fab3010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 --2- 172.21.15.19:0/1486979827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab30108a30 0x7fab30108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 --2- 172.21.15.19:0/1486979827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3006abb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 -- 172.21.15.19:0/1486979827 >> 172.21.15.19:0/1486979827 conn(0x7fab300788f0 msgr2=0x7fab30078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 -- 172.21.15.19:0/1486979827 shutdown_connections 2024-04-25T15:04:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.805+0000 7fab37a00700 1 -- 172.21.15.19:0/1486979827 wait complete. 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T15:04:38.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:38 smithi102 ceph-mon[33005]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.806+0000 7fab37a00700 1 Processor -- start 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.806+0000 7fab37a00700 1 -- start start 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab3006b0f0 0x7fab3019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 0x7fab3019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112c20 con 0x7fab3006b0f0 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112aa0 con 0x7fab3006a750 2024-04-25T15:04:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab37a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112da0 con 0x7fab30108a30 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab35a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 0x7fab3019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab35000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab35000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:38942/0 (socket says 172.21.15.19:38942) 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab2fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab3006b0f0 0x7fab3019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.807+0000 7fab35000700 1 -- 172.21.15.19:0/3697529934 learned_addr learned my addr 172.21.15.19:0/3697529934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 -- 172.21.15.19:0/3697529934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 msgr2=0x7fab3019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 -- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab3006b0f0 msgr2=0x7fab3019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab3006b0f0 0x7fab3019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 -- 172.21.15.19:0/3697529934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab301a03c0 con 0x7fab30108a30 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35000700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab35a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 0x7fab3019fc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab2800efc0 tx=0x7fab2800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab280092e0 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28004bb0 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab301a0650 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28004020 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.808+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a7f30 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.810+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab30104170 con 0x7fab30108a30 2024-04-25T15:04:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.811+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fab2801b030 con 0x7fab30108a30 2024-04-25T15:04:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.813+0000 7fab2d600700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 0x7fab180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.813+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fab280d03f0 con 0x7fab30108a30 2024-04-25T15:04:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.813+0000 7fab35000700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 0x7fab180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.814+0000 7fab35000700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 0x7fab180a6300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab24009680 tx=0x7fab24009260 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:38.817+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab28022020 con 0x7fab30108a30 2024-04-25T15:04:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.047+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab30062e80 con 0x7fab180a3e40 2024-04-25T15:04:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.059+0000 7fab2d600700 1 -- 172.21.15.19:0/3697529934 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257683 (secure 0 0 0) 0x7fab30062e80 con 0x7fab180a3e40 2024-04-25T15:04:39.064 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 msgr2=0x7fab180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 0x7fab180a6300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab24009680 tx=0x7fab24009260 comp rx=0 tx=0).stop 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 msgr2=0x7fab3019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 0x7fab3019fc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab2800efc0 tx=0x7fab2800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 shutdown_connections 2024-04-25T15:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fab180a3e40 0x7fab180a6300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab30108a30 0x7fab3019fc80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3006a750 0x7fab3019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 --2- 172.21.15.19:0/3697529934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab3006b0f0 0x7fab3019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 >> 172.21.15.19:0/3697529934 conn(0x7fab300788f0 msgr2=0x7fab3010df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 shutdown_connections 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:39.064+0000 7fab37a00700 1 -- 172.21.15.19:0/3697529934 wait complete. 2024-04-25T15:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.710 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:39 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:39 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: Updating smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:39 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:39.926 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-04-25T15:04:37.104839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1380094,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":901,"num_read_kb":867,"num_write":531,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390240,"kb_used_data":15744,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551584,"statfs":{"total":1151940427776,"available":1151540822016,"internally_reserved":0,"allocated":16121856,"data_stored":9885270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":362,"num_read_kb":445,"num_write":144,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.980838"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641948+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:04:29.641948+0000","last_peered":"2024-04-25T15:04:29.641948+0000","last_clean":"2024-04-25T15:04:29.641948+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:04:29.641948+0000","last_undegraded":"2024-04-25T15:04:29.641948+0000","last_fullsized":"2024-04-25T15:04:29.641948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507665+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:04:29.507665+0000","last_peered":"2024-04-25T15:04:29.507665+0000","last_clean":"2024-04-25T15:04:29.507665+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:04:29.507665+0000","last_undegraded":"2024-04-25T15:04:29.507665+0000","last_fullsized":"2024-04-25T15:04:29.507665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507262+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:04:29.507262+0000","last_peered":"2024-04-25T15:04:29.507262+0000","last_clean":"2024-04-25T15:04:29.507262+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:04:29.507262+0000","last_undegraded":"2024-04-25T15:04:29.507262+0000","last_fullsized":"2024-04-25T15:04:29.507262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083735+0000","last_change":"2024-04-25T15:02:57.474358+0000","last_active":"2024-04-25T15:04:29.083735+0000","last_peered":"2024-04-25T15:04:29.083735+0000","last_clean":"2024-04-25T15:04:29.083735+0000","last_became_active":"2024-04-25T15:02:57.473924+0000","last_became_peered":"2024-04-25T15:02:57.473924+0000","last_unstale":"2024-04-25T15:04:29.083735+0000","last_undegraded":"2024-04-25T15:04:29.083735+0000","last_fullsized":"2024-04-25T15:04:29.083735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.681108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205590+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:04:29.205590+0000","last_peered":"2024-04-25T15:04:29.205590+0000","last_clean":"2024-04-25T15:04:29.205590+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:04:29.205590+0000","last_undegraded":"2024-04-25T15:04:29.205590+0000","last_fullsized":"2024-04-25T15:04:29.205590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217157+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:04:29.217157+0000","last_peered":"2024-04-25T15:04:29.217157+0000","last_clean":"2024-04-25T15:04:29.217157+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:04:29.217157+0000","last_undegraded":"2024-04-25T15:04:29.217157+0000","last_fullsized":"2024-04-25T15:04:29.217157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354158+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:04:29.354158+0000","last_peered":"2024-04-25T15:04:29.354158+0000","last_clean":"2024-04-25T15:04:29.354158+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:04:29.354158+0000","last_undegraded":"2024-04-25T15:04:29.354158+0000","last_fullsized":"2024-04-25T15:04:29.354158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205987+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:04:29.205987+0000","last_peered":"2024-04-25T15:04:29.205987+0000","last_clean":"2024-04-25T15:04:29.205987+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:04:29.205987+0000","last_undegraded":"2024-04-25T15:04:29.205987+0000","last_fullsized":"2024-04-25T15:04:29.205987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325092+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:04:29.325092+0000","last_peered":"2024-04-25T15:04:29.325092+0000","last_clean":"2024-04-25T15:04:29.325092+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:04:29.325092+0000","last_undegraded":"2024-04-25T15:04:29.325092+0000","last_fullsized":"2024-04-25T15:04:29.325092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217524+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:04:29.217524+0000","last_peered":"2024-04-25T15:04:29.217524+0000","last_clean":"2024-04-25T15:04:29.217524+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:04:29.217524+0000","last_undegraded":"2024-04-25T15:04:29.217524+0000","last_fullsized":"2024-04-25T15:04:29.217524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354547+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:04:29.354547+0000","last_peered":"2024-04-25T15:04:29.354547+0000","last_clean":"2024-04-25T15:04:29.354547+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:04:29.354547+0000","last_undegraded":"2024-04-25T15:04:29.354547+0000","last_fullsized":"2024-04-25T15:04:29.354547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.078119+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:04:29.078119+0000","last_peered":"2024-04-25T15:04:29.078119+0000","last_clean":"2024-04-25T15:04:29.078119+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:04:29.078119+0000","last_undegraded":"2024-04-25T15:04:29.078119+0000","last_fullsized":"2024-04-25T15:04:29.078119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642134+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:04:29.642134+0000","last_peered":"2024-04-25T15:04:29.642134+0000","last_clean":"2024-04-25T15:04:29.642134+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:04:29.642134+0000","last_undegraded":"2024-04-25T15:04:29.642134+0000","last_fullsized":"2024-04-25T15:04:29.642134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507345+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:04:29.507345+0000","last_peered":"2024-04-25T15:04:29.507345+0000","last_clean":"2024-04-25T15:04:29.507345+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:04:29.507345+0000","last_undegraded":"2024-04-25T15:04:29.507345+0000","last_fullsized":"2024-04-25T15:04:29.507345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507098+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:04:29.507098+0000","last_peered":"2024-04-25T15:04:29.507098+0000","last_clean":"2024-04-25T15:04:29.507098+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:04:29.507098+0000","last_undegraded":"2024-04-25T15:04:29.507098+0000","last_fullsized":"2024-04-25T15:04:29.507098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324603+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:04:29.324603+0000","last_peered":"2024-04-25T15:04:29.324603+0000","last_clean":"2024-04-25T15:04:29.324603+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:04:29.324603+0000","last_undegraded":"2024-04-25T15:04:29.324603+0000","last_fullsized":"2024-04-25T15:04:29.324603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642011+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:04:29.642011+0000","last_peered":"2024-04-25T15:04:29.642011+0000","last_clean":"2024-04-25T15:04:29.642011+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:04:29.642011+0000","last_undegraded":"2024-04-25T15:04:29.642011+0000","last_fullsized":"2024-04-25T15:04:29.642011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083738+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:04:29.083738+0000","last_peered":"2024-04-25T15:04:29.083738+0000","last_clean":"2024-04-25T15:04:29.083738+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:04:29.083738+0000","last_undegraded":"2024-04-25T15:04:29.083738+0000","last_fullsized":"2024-04-25T15:04:29.083738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079371+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:04:29.079371+0000","last_peered":"2024-04-25T15:04:29.079371+0000","last_clean":"2024-04-25T15:04:29.079371+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:04:29.079371+0000","last_undegraded":"2024-04-25T15:04:29.079371+0000","last_fullsized":"2024-04-25T15:04:29.079371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.085077+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:04:29.085077+0000","last_peered":"2024-04-25T15:04:29.085077+0000","last_clean":"2024-04-25T15:04:29.085077+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:04:29.085077+0000","last_undegraded":"2024-04-25T15:04:29.085077+0000","last_fullsized":"2024-04-25T15:04:29.085077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205959+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:04:29.205959+0000","last_peered":"2024-04-25T15:04:29.205959+0000","last_clean":"2024-04-25T15:04:29.205959+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:04:29.205959+0000","last_undegraded":"2024-04-25T15:04:29.205959+0000","last_fullsized":"2024-04-25T15:04:29.205959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206088+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:04:29.206088+0000","last_peered":"2024-04-25T15:04:29.206088+0000","last_clean":"2024-04-25T15:04:29.206088+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:04:29.206088+0000","last_undegraded":"2024-04-25T15:04:29.206088+0000","last_fullsized":"2024-04-25T15:04:29.206088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507821+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:04:29.507821+0000","last_peered":"2024-04-25T15:04:29.507821+0000","last_clean":"2024-04-25T15:04:29.507821+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:04:29.507821+0000","last_undegraded":"2024-04-25T15:04:29.507821+0000","last_fullsized":"2024-04-25T15:04:29.507821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205973+0000","last_change":"2024-04-25T15:02:57.476182+0000","last_active":"2024-04-25T15:04:29.205973+0000","last_peered":"2024-04-25T15:04:29.205973+0000","last_clean":"2024-04-25T15:04:29.205973+0000","last_became_active":"2024-04-25T15:02:57.472985+0000","last_became_peered":"2024-04-25T15:02:57.472985+0000","last_unstale":"2024-04-25T15:04:29.205973+0000","last_undegraded":"2024-04-25T15:04:29.205973+0000","last_fullsized":"2024-04-25T15:04:29.205973+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.669296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325105+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:04:29.325105+0000","last_peered":"2024-04-25T15:04:29.325105+0000","last_clean":"2024-04-25T15:04:29.325105+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:04:29.325105+0000","last_undegraded":"2024-04-25T15:04:29.325105+0000","last_fullsized":"2024-04-25T15:04:29.325105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205852+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:04:29.205852+0000","last_peered":"2024-04-25T15:04:29.205852+0000","last_clean":"2024-04-25T15:04:29.205852+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:04:29.205852+0000","last_undegraded":"2024-04-25T15:04:29.205852+0000","last_fullsized":"2024-04-25T15:04:29.205852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213970+0000","last_change":"2024-04-25T15:02:57.471043+0000","last_active":"2024-04-25T15:04:29.213970+0000","last_peered":"2024-04-25T15:04:29.213970+0000","last_clean":"2024-04-25T15:04:29.213970+0000","last_became_active":"2024-04-25T15:02:57.470670+0000","last_became_peered":"2024-04-25T15:02:57.470670+0000","last_unstale":"2024-04-25T15:04:29.213970+0000","last_undegraded":"2024-04-25T15:04:29.213970+0000","last_fullsized":"2024-04-25T15:04:29.213970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.243201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213843+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:04:29.213843+0000","last_peered":"2024-04-25T15:04:29.213843+0000","last_clean":"2024-04-25T15:04:29.213843+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:04:29.213843+0000","last_undegraded":"2024-04-25T15:04:29.213843+0000","last_fullsized":"2024-04-25T15:04:29.213843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205929+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:04:29.205929+0000","last_peered":"2024-04-25T15:04:29.205929+0000","last_clean":"2024-04-25T15:04:29.205929+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:04:29.205929+0000","last_undegraded":"2024-04-25T15:04:29.205929+0000","last_fullsized":"2024-04-25T15:04:29.205929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217119+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:04:29.217119+0000","last_peered":"2024-04-25T15:04:29.217119+0000","last_clean":"2024-04-25T15:04:29.217119+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:04:29.217119+0000","last_undegraded":"2024-04-25T15:04:29.217119+0000","last_fullsized":"2024-04-25T15:04:29.217119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213785+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:04:29.213785+0000","last_peered":"2024-04-25T15:04:29.213785+0000","last_clean":"2024-04-25T15:04:29.213785+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:04:29.213785+0000","last_undegraded":"2024-04-25T15:04:29.213785+0000","last_fullsized":"2024-04-25T15:04:29.213785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642227+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:04:29.642227+0000","last_peered":"2024-04-25T15:04:29.642227+0000","last_clean":"2024-04-25T15:04:29.642227+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:04:29.642227+0000","last_undegraded":"2024-04-25T15:04:29.642227+0000","last_fullsized":"2024-04-25T15:04:29.642227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205603+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:04:29.205603+0000","last_peered":"2024-04-25T15:04:29.205603+0000","last_clean":"2024-04-25T15:04:29.205603+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:04:29.205603+0000","last_undegraded":"2024-04-25T15:04:29.205603+0000","last_fullsized":"2024-04-25T15:04:29.205603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213605+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:04:29.213605+0000","last_peered":"2024-04-25T15:04:29.213605+0000","last_clean":"2024-04-25T15:04:29.213605+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:04:29.213605+0000","last_undegraded":"2024-04-25T15:04:29.213605+0000","last_fullsized":"2024-04-25T15:04:29.213605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083091+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:04:29.083091+0000","last_peered":"2024-04-25T15:04:29.083091+0000","last_clean":"2024-04-25T15:04:29.083091+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:04:29.083091+0000","last_undegraded":"2024-04-25T15:04:29.083091+0000","last_fullsized":"2024-04-25T15:04:29.083091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217382+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:04:29.217382+0000","last_peered":"2024-04-25T15:04:29.217382+0000","last_clean":"2024-04-25T15:04:29.217382+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:04:29.217382+0000","last_undegraded":"2024-04-25T15:04:29.217382+0000","last_fullsized":"2024-04-25T15:04:29.217382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325069+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:04:29.325069+0000","last_peered":"2024-04-25T15:04:29.325069+0000","last_clean":"2024-04-25T15:04:29.325069+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:04:29.325069+0000","last_undegraded":"2024-04-25T15:04:29.325069+0000","last_fullsized":"2024-04-25T15:04:29.325069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354452+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:04:29.354452+0000","last_peered":"2024-04-25T15:04:29.354452+0000","last_clean":"2024-04-25T15:04:29.354452+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:04:29.354452+0000","last_undegraded":"2024-04-25T15:04:29.354452+0000","last_fullsized":"2024-04-25T15:04:29.354452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507878+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:04:29.507878+0000","last_peered":"2024-04-25T15:04:29.507878+0000","last_clean":"2024-04-25T15:04:29.507878+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:04:29.507878+0000","last_undegraded":"2024-04-25T15:04:29.507878+0000","last_fullsized":"2024-04-25T15:04:29.507878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.214112+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:04:29.214112+0000","last_peered":"2024-04-25T15:04:29.214112+0000","last_clean":"2024-04-25T15:04:29.214112+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:04:29.214112+0000","last_undegraded":"2024-04-25T15:04:29.214112+0000","last_fullsized":"2024-04-25T15:04:29.214112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325015+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:04:29.325015+0000","last_peered":"2024-04-25T15:04:29.325015+0000","last_clean":"2024-04-25T15:04:29.325015+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:04:29.325015+0000","last_undegraded":"2024-04-25T15:04:29.325015+0000","last_fullsized":"2024-04-25T15:04:29.325015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":371,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:34.629178+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:04:34.629178+0000","last_peered":"2024-04-25T15:04:34.629178+0000","last_clean":"2024-04-25T15:04:34.629178+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:04:34.629178+0000","last_undegraded":"2024-04-25T15:04:34.629178+0000","last_fullsized":"2024-04-25T15:04:34.629178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67246,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":314,"num_read_kb":223,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079551+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:04:29.079551+0000","last_peered":"2024-04-25T15:04:29.079551+0000","last_clean":"2024-04-25T15:04:29.079551+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:04:29.079551+0000","last_undegraded":"2024-04-25T15:04:29.079551+0000","last_fullsized":"2024-04-25T15:04:29.079551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641849+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:04:29.641849+0000","last_peered":"2024-04-25T15:04:29.641849+0000","last_clean":"2024-04-25T15:04:29.641849+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:04:29.641849+0000","last_undegraded":"2024-04-25T15:04:29.641849+0000","last_fullsized":"2024-04-25T15:04:29.641849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213964+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:04:29.213964+0000","last_peered":"2024-04-25T15:04:29.213964+0000","last_clean":"2024-04-25T15:04:29.213964+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:04:29.213964+0000","last_undegraded":"2024-04-25T15:04:29.213964+0000","last_fullsized":"2024-04-25T15:04:29.213964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354351+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:04:29.354351+0000","last_peered":"2024-04-25T15:04:29.354351+0000","last_clean":"2024-04-25T15:04:29.354351+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:04:29.354351+0000","last_undegraded":"2024-04-25T15:04:29.354351+0000","last_fullsized":"2024-04-25T15:04:29.354351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077852+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:04:29.077852+0000","last_peered":"2024-04-25T15:04:29.077852+0000","last_clean":"2024-04-25T15:04:29.077852+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:04:29.077852+0000","last_undegraded":"2024-04-25T15:04:29.077852+0000","last_fullsized":"2024-04-25T15:04:29.077852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354095+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:04:29.354095+0000","last_peered":"2024-04-25T15:04:29.354095+0000","last_clean":"2024-04-25T15:04:29.354095+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:04:29.354095+0000","last_undegraded":"2024-04-25T15:04:29.354095+0000","last_fullsized":"2024-04-25T15:04:29.354095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205561+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:04:29.205561+0000","last_peered":"2024-04-25T15:04:29.205561+0000","last_clean":"2024-04-25T15:04:29.205561+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:04:29.205561+0000","last_undegraded":"2024-04-25T15:04:29.205561+0000","last_fullsized":"2024-04-25T15:04:29.205561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213715+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:04:29.213715+0000","last_peered":"2024-04-25T15:04:29.213715+0000","last_clean":"2024-04-25T15:04:29.213715+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:04:29.213715+0000","last_undegraded":"2024-04-25T15:04:29.213715+0000","last_fullsized":"2024-04-25T15:04:29.213715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641863+0000","last_change":"2024-04-25T15:02:57.473783+0000","last_active":"2024-04-25T15:04:29.641863+0000","last_peered":"2024-04-25T15:04:29.641863+0000","last_clean":"2024-04-25T15:04:29.641863+0000","last_became_active":"2024-04-25T15:02:57.468466+0000","last_became_peered":"2024-04-25T15:02:57.468466+0000","last_unstale":"2024-04-25T15:04:29.641863+0000","last_undegraded":"2024-04-25T15:04:29.641863+0000","last_fullsized":"2024-04-25T15:04:29.641863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:41.182777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641910+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:04:29.641910+0000","last_peered":"2024-04-25T15:04:29.641910+0000","last_clean":"2024-04-25T15:04:29.641910+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:04:29.641910+0000","last_undegraded":"2024-04-25T15:04:29.641910+0000","last_fullsized":"2024-04-25T15:04:29.641910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083393+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:04:29.083393+0000","last_peered":"2024-04-25T15:04:29.083393+0000","last_clean":"2024-04-25T15:04:29.083393+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:04:29.083393+0000","last_undegraded":"2024-04-25T15:04:29.083393+0000","last_fullsized":"2024-04-25T15:04:29.083393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217387+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:04:29.217387+0000","last_peered":"2024-04-25T15:04:29.217387+0000","last_clean":"2024-04-25T15:04:29.217387+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:04:29.217387+0000","last_undegraded":"2024-04-25T15:04:29.217387+0000","last_fullsized":"2024-04-25T15:04:29.217387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354189+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:04:29.354189+0000","last_peered":"2024-04-25T15:04:29.354189+0000","last_clean":"2024-04-25T15:04:29.354189+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:04:29.354189+0000","last_undegraded":"2024-04-25T15:04:29.354189+0000","last_fullsized":"2024-04-25T15:04:29.354189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217231+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:04:29.217231+0000","last_peered":"2024-04-25T15:04:29.217231+0000","last_clean":"2024-04-25T15:04:29.217231+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:04:29.217231+0000","last_undegraded":"2024-04-25T15:04:29.217231+0000","last_fullsized":"2024-04-25T15:04:29.217231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213699+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:04:29.213699+0000","last_peered":"2024-04-25T15:04:29.213699+0000","last_clean":"2024-04-25T15:04:29.213699+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:04:29.213699+0000","last_undegraded":"2024-04-25T15:04:29.213699+0000","last_fullsized":"2024-04-25T15:04:29.213699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641750+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:04:29.641750+0000","last_peered":"2024-04-25T15:04:29.641750+0000","last_clean":"2024-04-25T15:04:29.641750+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:04:29.641750+0000","last_undegraded":"2024-04-25T15:04:29.641750+0000","last_fullsized":"2024-04-25T15:04:29.641750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205739+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:04:29.205739+0000","last_peered":"2024-04-25T15:04:29.205739+0000","last_clean":"2024-04-25T15:04:29.205739+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:04:29.205739+0000","last_undegraded":"2024-04-25T15:04:29.205739+0000","last_fullsized":"2024-04-25T15:04:29.205739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077898+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:04:29.077898+0000","last_peered":"2024-04-25T15:04:29.077898+0000","last_clean":"2024-04-25T15:04:29.077898+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:04:29.077898+0000","last_undegraded":"2024-04-25T15:04:29.077898+0000","last_fullsized":"2024-04-25T15:04:29.077898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":401,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.227382+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:04:31.227382+0000","last_peered":"2024-04-25T15:04:31.227382+0000","last_clean":"2024-04-25T15:04:31.227382+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:04:31.227382+0000","last_undegraded":"2024-04-25T15:04:31.227382+0000","last_fullsized":"2024-04-25T15:04:31.227382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641602+0000","last_change":"2024-04-25T15:02:57.474003+0000","last_active":"2024-04-25T15:04:29.641602+0000","last_peered":"2024-04-25T15:04:29.641602+0000","last_clean":"2024-04-25T15:04:29.641602+0000","last_became_active":"2024-04-25T15:02:57.473443+0000","last_became_peered":"2024-04-25T15:02:57.473443+0000","last_unstale":"2024-04-25T15:04:29.641602+0000","last_undegraded":"2024-04-25T15:04:29.641602+0000","last_fullsized":"2024-04-25T15:04:29.641602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:30.170675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354286+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:04:29.354286+0000","last_peered":"2024-04-25T15:04:29.354286+0000","last_clean":"2024-04-25T15:04:29.354286+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:04:29.354286+0000","last_undegraded":"2024-04-25T15:04:29.354286+0000","last_fullsized":"2024-04-25T15:04:29.354286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641817+0000","last_change":"2024-04-25T15:02:57.529669+0000","last_active":"2024-04-25T15:04:29.641817+0000","last_peered":"2024-04-25T15:04:29.641817+0000","last_clean":"2024-04-25T15:04:29.641817+0000","last_became_active":"2024-04-25T15:02:57.529390+0000","last_became_peered":"2024-04-25T15:02:57.529390+0000","last_unstale":"2024-04-25T15:04:29.641817+0000","last_undegraded":"2024-04-25T15:04:29.641817+0000","last_fullsized":"2024-04-25T15:04:29.641817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:36.255304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642335+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:04:29.642335+0000","last_peered":"2024-04-25T15:04:29.642335+0000","last_clean":"2024-04-25T15:04:29.642335+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:04:29.642335+0000","last_undegraded":"2024-04-25T15:04:29.642335+0000","last_fullsized":"2024-04-25T15:04:29.642335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'98","reported_seq":212,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.812284+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:04:31.812284+0000","last_peered":"2024-04-25T15:04:31.812284+0000","last_clean":"2024-04-25T15:04:31.812284+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:04:31.812284+0000","last_undegraded":"2024-04-25T15:04:31.812284+0000","last_fullsized":"2024-04-25T15:04:31.812284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":180,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206049+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:04:29.206049+0000","last_peered":"2024-04-25T15:04:29.206049+0000","last_clean":"2024-04-25T15:04:29.206049+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:04:29.206049+0000","last_undegraded":"2024-04-25T15:04:29.206049+0000","last_fullsized":"2024-04-25T15:04:29.206049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077925+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:04:29.077925+0000","last_peered":"2024-04-25T15:04:29.077925+0000","last_clean":"2024-04-25T15:04:29.077925+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:04:29.077925+0000","last_undegraded":"2024-04-25T15:04:29.077925+0000","last_fullsized":"2024-04-25T15:04:29.077925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507811+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:04:29.507811+0000","last_peered":"2024-04-25T15:04:29.507811+0000","last_clean":"2024-04-25T15:04:29.507811+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:04:29.507811+0000","last_undegraded":"2024-04-25T15:04:29.507811+0000","last_fullsized":"2024-04-25T15:04:29.507811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324508+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:04:29.324508+0000","last_peered":"2024-04-25T15:04:29.324508+0000","last_clean":"2024-04-25T15:04:29.324508+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:04:29.324508+0000","last_undegraded":"2024-04-25T15:04:29.324508+0000","last_fullsized":"2024-04-25T15:04:29.324508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205471+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:04:29.205471+0000","last_peered":"2024-04-25T15:04:29.205471+0000","last_clean":"2024-04-25T15:04:29.205471+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:04:29.205471+0000","last_undegraded":"2024-04-25T15:04:29.205471+0000","last_fullsized":"2024-04-25T15:04:29.205471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206025+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:04:29.206025+0000","last_peered":"2024-04-25T15:04:29.206025+0000","last_clean":"2024-04-25T15:04:29.206025+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:04:29.206025+0000","last_undegraded":"2024-04-25T15:04:29.206025+0000","last_fullsized":"2024-04-25T15:04:29.206025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641669+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:04:29.641669+0000","last_peered":"2024-04-25T15:04:29.641669+0000","last_clean":"2024-04-25T15:04:29.641669+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:04:29.641669+0000","last_undegraded":"2024-04-25T15:04:29.641669+0000","last_fullsized":"2024-04-25T15:04:29.641669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079203+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:04:29.079203+0000","last_peered":"2024-04-25T15:04:29.079203+0000","last_clean":"2024-04-25T15:04:29.079203+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:04:29.079203+0000","last_undegraded":"2024-04-25T15:04:29.079203+0000","last_fullsized":"2024-04-25T15:04:29.079203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205734+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:04:29.205734+0000","last_peered":"2024-04-25T15:04:29.205734+0000","last_clean":"2024-04-25T15:04:29.205734+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:04:29.205734+0000","last_undegraded":"2024-04-25T15:04:29.205734+0000","last_fullsized":"2024-04-25T15:04:29.205734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205592+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:04:29.205592+0000","last_peered":"2024-04-25T15:04:29.205592+0000","last_clean":"2024-04-25T15:04:29.205592+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:04:29.205592+0000","last_undegraded":"2024-04-25T15:04:29.205592+0000","last_fullsized":"2024-04-25T15:04:29.205592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354285+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:04:29.354285+0000","last_peered":"2024-04-25T15:04:29.354285+0000","last_clean":"2024-04-25T15:04:29.354285+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:04:29.354285+0000","last_undegraded":"2024-04-25T15:04:29.354285+0000","last_fullsized":"2024-04-25T15:04:29.354285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324613+0000","last_change":"2024-04-25T15:02:57.529816+0000","last_active":"2024-04-25T15:04:29.324613+0000","last_peered":"2024-04-25T15:04:29.324613+0000","last_clean":"2024-04-25T15:04:29.324613+0000","last_became_active":"2024-04-25T15:02:57.529442+0000","last_became_peered":"2024-04-25T15:02:57.529442+0000","last_unstale":"2024-04-25T15:04:29.324613+0000","last_undegraded":"2024-04-25T15:04:29.324613+0000","last_fullsized":"2024-04-25T15:04:29.324613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.143614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354562+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:04:29.354562+0000","last_peered":"2024-04-25T15:04:29.354562+0000","last_clean":"2024-04-25T15:04:29.354562+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:04:29.354562+0000","last_undegraded":"2024-04-25T15:04:29.354562+0000","last_fullsized":"2024-04-25T15:04:29.354562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'65","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.325885+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:04:31.325885+0000","last_peered":"2024-04-25T15:04:31.325885+0000","last_clean":"2024-04-25T15:04:31.325885+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:04:31.325885+0000","last_undegraded":"2024-04-25T15:04:31.325885+0000","last_fullsized":"2024-04-25T15:04:31.325885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354311+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:04:29.354311+0000","last_peered":"2024-04-25T15:04:29.354311+0000","last_clean":"2024-04-25T15:04:29.354311+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:04:29.354311+0000","last_undegraded":"2024-04-25T15:04:29.354311+0000","last_fullsized":"2024-04-25T15:04:29.354311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642550+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:04:29.642550+0000","last_peered":"2024-04-25T15:04:29.642550+0000","last_clean":"2024-04-25T15:04:29.642550+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:04:29.642550+0000","last_undegraded":"2024-04-25T15:04:29.642550+0000","last_fullsized":"2024-04-25T15:04:29.642550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354512+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:04:29.354512+0000","last_peered":"2024-04-25T15:04:29.354512+0000","last_clean":"2024-04-25T15:04:29.354512+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:04:29.354512+0000","last_undegraded":"2024-04-25T15:04:29.354512+0000","last_fullsized":"2024-04-25T15:04:29.354512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641721+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:04:29.641721+0000","last_peered":"2024-04-25T15:04:29.641721+0000","last_clean":"2024-04-25T15:04:29.641721+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:04:29.641721+0000","last_undegraded":"2024-04-25T15:04:29.641721+0000","last_fullsized":"2024-04-25T15:04:29.641721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079375+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:04:29.079375+0000","last_peered":"2024-04-25T15:04:29.079375+0000","last_clean":"2024-04-25T15:04:29.079375+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:04:29.079375+0000","last_undegraded":"2024-04-25T15:04:29.079375+0000","last_fullsized":"2024-04-25T15:04:29.079375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079220+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:04:29.079220+0000","last_peered":"2024-04-25T15:04:29.079220+0000","last_clean":"2024-04-25T15:04:29.079220+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:04:29.079220+0000","last_undegraded":"2024-04-25T15:04:29.079220+0000","last_fullsized":"2024-04-25T15:04:29.079220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507354+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:04:29.507354+0000","last_peered":"2024-04-25T15:04:29.507354+0000","last_clean":"2024-04-25T15:04:29.507354+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:04:29.507354+0000","last_undegraded":"2024-04-25T15:04:29.507354+0000","last_fullsized":"2024-04-25T15:04:29.507354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642445+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:04:29.642445+0000","last_peered":"2024-04-25T15:04:29.642445+0000","last_clean":"2024-04-25T15:04:29.642445+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:04:29.642445+0000","last_undegraded":"2024-04-25T15:04:29.642445+0000","last_fullsized":"2024-04-25T15:04:29.642445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641969+0000","last_change":"2024-04-25T15:02:57.470460+0000","last_active":"2024-04-25T15:04:29.641969+0000","last_peered":"2024-04-25T15:04:29.641969+0000","last_clean":"2024-04-25T15:04:29.641969+0000","last_became_active":"2024-04-25T15:02:57.470013+0000","last_became_peered":"2024-04-25T15:02:57.470013+0000","last_unstale":"2024-04-25T15:04:29.641969+0000","last_undegraded":"2024-04-25T15:04:29.641969+0000","last_fullsized":"2024-04-25T15:04:29.641969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:07.351115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079804+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:04:29.079804+0000","last_peered":"2024-04-25T15:04:29.079804+0000","last_clean":"2024-04-25T15:04:29.079804+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:04:29.079804+0000","last_undegraded":"2024-04-25T15:04:29.079804+0000","last_fullsized":"2024-04-25T15:04:29.079804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213572+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:04:29.213572+0000","last_peered":"2024-04-25T15:04:29.213572+0000","last_clean":"2024-04-25T15:04:29.213572+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:04:29.213572+0000","last_undegraded":"2024-04-25T15:04:29.213572+0000","last_fullsized":"2024-04-25T15:04:29.213572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205767+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:04:29.205767+0000","last_peered":"2024-04-25T15:04:29.205767+0000","last_clean":"2024-04-25T15:04:29.205767+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:04:29.205767+0000","last_undegraded":"2024-04-25T15:04:29.205767+0000","last_fullsized":"2024-04-25T15:04:29.205767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324892+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:04:29.324892+0000","last_peered":"2024-04-25T15:04:29.324892+0000","last_clean":"2024-04-25T15:04:29.324892+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:04:29.324892+0000","last_undegraded":"2024-04-25T15:04:29.324892+0000","last_fullsized":"2024-04-25T15:04:29.324892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642103+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:04:29.642103+0000","last_peered":"2024-04-25T15:04:29.642103+0000","last_clean":"2024-04-25T15:04:29.642103+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:04:29.642103+0000","last_undegraded":"2024-04-25T15:04:29.642103+0000","last_fullsized":"2024-04-25T15:04:29.642103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641698+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:04:29.641698+0000","last_peered":"2024-04-25T15:04:29.641698+0000","last_clean":"2024-04-25T15:04:29.641698+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:04:29.641698+0000","last_undegraded":"2024-04-25T15:04:29.641698+0000","last_fullsized":"2024-04-25T15:04:29.641698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":789492,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":664,"num_read_kb":537,"num_write":328,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":2368476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285611,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29096,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716056,"statfs":{"total":95995035648,"available":95965241344,"internally_reserved":0,"allocated":2007040,"data_stored":1464906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28368,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716784,"statfs":{"total":95995035648,"available":95965986816,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1069816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":545995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":546029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28824,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716328,"statfs":{"total":95995035648,"available":95965519872,"internally_reserved":0,"allocated":1728512,"data_stored":1200535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1929216,"data_stored":1397648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":395672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:04:39.929 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T15:04:39.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:04:40.410 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:04:40.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:40.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T15:04:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:40 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.760 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:40.760 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:40.761 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T15:04:40.761 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.761 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.761 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:40 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:04:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T15:04:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:40 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:41.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:41 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.729 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:41.729 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:41.730 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.730 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.730 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:41.730 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:41.730 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:41 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: Updating smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:04:41.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:41.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:41 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:04:42.721 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:42 smithi019 ceph-mon[28090]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-25T15:04:42.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:42 smithi083 ceph-mon[32919]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-25T15:04:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:42 smithi102 ceph-mon[33005]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-25T15:04:43.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.105+0000 7f8e8c600700 1 -- 172.21.15.19:0/676115325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8410f440 msgr2=0x7f8e84111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:43.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.105+0000 7f8e8c600700 1 --2- 172.21.15.19:0/676115325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8410f440 0x7f8e84111830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009f90 tx=0x7f8e7c00b330 comp rx=0 tx=0).stop 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 -- 172.21.15.19:0/676115325 shutdown_connections 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 --2- 172.21.15.19:0/676115325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e8410f440 0x7f8e84111830 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 --2- 172.21.15.19:0/676115325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84074c70 0x7f8e84075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 --2- 172.21.15.19:0/676115325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e84075590 0x7f8e8410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 -- 172.21.15.19:0/676115325 >> 172.21.15.19:0/676115325 conn(0x7f8e84079ba0 msgr2=0x7f8e84079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 -- 172.21.15.19:0/676115325 shutdown_connections 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.106+0000 7f8e8c600700 1 -- 172.21.15.19:0/676115325 wait complete. 2024-04-25T15:04:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.107+0000 7f8e8c600700 1 Processor -- start 2024-04-25T15:04:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 -- start start 2024-04-25T15:04:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 0x7f8e8419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 0x7f8e8419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84114e10 con 0x7f8e8410f440 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84114c90 con 0x7f8e84075590 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e8c600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84114f90 con 0x7f8e84074c70 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e89c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e89c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:45620/0 (socket says 172.21.15.19:45620) 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.108+0000 7f8e89c00700 1 -- 172.21.15.19:0/878643141 learned_addr learned my addr 172.21.15.19:0/878643141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e8a600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 0x7f8e8419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89200700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 0x7f8e8419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 -- 172.21.15.19:0/878643141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 msgr2=0x7f8e8419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 0x7f8e8419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 -- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 msgr2=0x7f8e8419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 0x7f8e8419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 -- 172.21.15.19:0/878643141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e841a03f0 con 0x7f8e84074c70 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89200700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 0x7f8e8419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e8a600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 0x7f8e8419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.109+0000 7f8e89c00700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e9a0 tx=0x7f8e7800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.110+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800c9c0 con 0x7f8e84074c70 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.110+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7800cb60 con 0x7f8e84074c70 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.110+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e841a06e0 con 0x7f8e84074c70 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.110+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78018680 con 0x7f8e84074c70 2024-04-25T15:04:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.110+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a7f60 con 0x7f8e84074c70 2024-04-25T15:04:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.111+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e84069730 con 0x7f8e84074c70 2024-04-25T15:04:43.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.113+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8e78010430 con 0x7f8e84074c70 2024-04-25T15:04:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.114+0000 7f8e82000700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 0x7f8e6c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.115+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8e780d0c70 con 0x7f8e84074c70 2024-04-25T15:04:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.115+0000 7f8e89200700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 0x7f8e6c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.115+0000 7f8e89200700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 0x7f8e6c0a6300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e84101bb0 tx=0x7f8e70009310 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.119+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e7809d170 con 0x7f8e84074c70 2024-04-25T15:04:43.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.348+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e8419c740 con 0x7f8e6c0a3e40 2024-04-25T15:04:43.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.361+0000 7f8e82000700 1 -- 172.21.15.19:0/878643141 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257685 (secure 0 0 0) 0x7f8e8419c740 con 0x7f8e6c0a3e40 2024-04-25T15:04:43.370 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 msgr2=0x7f8e6c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 0x7f8e6c0a6300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e84101bb0 tx=0x7f8e70009310 comp rx=0 tx=0).stop 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 msgr2=0x7f8e8419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e9a0 tx=0x7f8e7800ecb0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 shutdown_connections 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8e6c0a3e40 0x7f8e6c0a6300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e8410f440 0x7f8e8419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e84074c70 0x7f8e8419b3e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 --2- 172.21.15.19:0/878643141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e84075590 0x7f8e8419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 >> 172.21.15.19:0/878643141 conn(0x7f8e84079ba0 msgr2=0x7f8e84100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 shutdown_connections 2024-04-25T15:04:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:43.367+0000 7f8e8c600700 1 -- 172.21.15.19:0/878643141 wait complete. 2024-04-25T15:04:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T15:04:44.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T15:04:43.107476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1380094,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":911,"num_read_kb":872,"num_write":531,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390240,"kb_used_data":15744,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551584,"statfs":{"total":1151940427776,"available":1151540822016,"internally_reserved":0,"allocated":16121856,"data_stored":9885270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":450,"num_write":144,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004301"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641948+0000","last_change":"2024-04-25T15:02:00.945145+0000","last_active":"2024-04-25T15:04:29.641948+0000","last_peered":"2024-04-25T15:04:29.641948+0000","last_clean":"2024-04-25T15:04:29.641948+0000","last_became_active":"2024-04-25T15:02:00.944799+0000","last_became_peered":"2024-04-25T15:02:00.944799+0000","last_unstale":"2024-04-25T15:04:29.641948+0000","last_undegraded":"2024-04-25T15:04:29.641948+0000","last_fullsized":"2024-04-25T15:04:29.641948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:25.922561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507665+0000","last_change":"2024-04-25T15:02:00.941176+0000","last_active":"2024-04-25T15:04:29.507665+0000","last_peered":"2024-04-25T15:04:29.507665+0000","last_clean":"2024-04-25T15:04:29.507665+0000","last_became_active":"2024-04-25T15:02:00.940700+0000","last_became_peered":"2024-04-25T15:02:00.940700+0000","last_unstale":"2024-04-25T15:04:29.507665+0000","last_undegraded":"2024-04-25T15:04:29.507665+0000","last_fullsized":"2024-04-25T15:04:29.507665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:23.962804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507262+0000","last_change":"2024-04-25T15:02:00.948863+0000","last_active":"2024-04-25T15:04:29.507262+0000","last_peered":"2024-04-25T15:04:29.507262+0000","last_clean":"2024-04-25T15:04:29.507262+0000","last_became_active":"2024-04-25T15:02:00.948114+0000","last_became_peered":"2024-04-25T15:02:00.948114+0000","last_unstale":"2024-04-25T15:04:29.507262+0000","last_undegraded":"2024-04-25T15:04:29.507262+0000","last_fullsized":"2024-04-25T15:04:29.507262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:36.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083735+0000","last_change":"2024-04-25T15:02:57.474358+0000","last_active":"2024-04-25T15:04:29.083735+0000","last_peered":"2024-04-25T15:04:29.083735+0000","last_clean":"2024-04-25T15:04:29.083735+0000","last_became_active":"2024-04-25T15:02:57.473924+0000","last_became_peered":"2024-04-25T15:02:57.473924+0000","last_unstale":"2024-04-25T15:04:29.083735+0000","last_undegraded":"2024-04-25T15:04:29.083735+0000","last_fullsized":"2024-04-25T15:04:29.083735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.681108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205590+0000","last_change":"2024-04-25T15:02:00.963657+0000","last_active":"2024-04-25T15:04:29.205590+0000","last_peered":"2024-04-25T15:04:29.205590+0000","last_clean":"2024-04-25T15:04:29.205590+0000","last_became_active":"2024-04-25T15:02:00.963243+0000","last_became_peered":"2024-04-25T15:02:00.963243+0000","last_unstale":"2024-04-25T15:04:29.205590+0000","last_undegraded":"2024-04-25T15:04:29.205590+0000","last_fullsized":"2024-04-25T15:04:29.205590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:27.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217157+0000","last_change":"2024-04-25T15:02:00.949202+0000","last_active":"2024-04-25T15:04:29.217157+0000","last_peered":"2024-04-25T15:04:29.217157+0000","last_clean":"2024-04-25T15:04:29.217157+0000","last_became_active":"2024-04-25T15:02:00.948818+0000","last_became_peered":"2024-04-25T15:02:00.948818+0000","last_unstale":"2024-04-25T15:04:29.217157+0000","last_undegraded":"2024-04-25T15:04:29.217157+0000","last_fullsized":"2024-04-25T15:04:29.217157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.340187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354158+0000","last_change":"2024-04-25T15:02:00.949597+0000","last_active":"2024-04-25T15:04:29.354158+0000","last_peered":"2024-04-25T15:04:29.354158+0000","last_clean":"2024-04-25T15:04:29.354158+0000","last_became_active":"2024-04-25T15:02:00.949259+0000","last_became_peered":"2024-04-25T15:02:00.949259+0000","last_unstale":"2024-04-25T15:04:29.354158+0000","last_undegraded":"2024-04-25T15:04:29.354158+0000","last_fullsized":"2024-04-25T15:04:29.354158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.197463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205987+0000","last_change":"2024-04-25T15:02:00.956308+0000","last_active":"2024-04-25T15:04:29.205987+0000","last_peered":"2024-04-25T15:04:29.205987+0000","last_clean":"2024-04-25T15:04:29.205987+0000","last_became_active":"2024-04-25T15:02:00.955369+0000","last_became_peered":"2024-04-25T15:02:00.955369+0000","last_unstale":"2024-04-25T15:04:29.205987+0000","last_undegraded":"2024-04-25T15:04:29.205987+0000","last_fullsized":"2024-04-25T15:04:29.205987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:09.614316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325092+0000","last_change":"2024-04-25T15:02:00.943244+0000","last_active":"2024-04-25T15:04:29.325092+0000","last_peered":"2024-04-25T15:04:29.325092+0000","last_clean":"2024-04-25T15:04:29.325092+0000","last_became_active":"2024-04-25T15:02:00.942922+0000","last_became_peered":"2024-04-25T15:02:00.942922+0000","last_unstale":"2024-04-25T15:04:29.325092+0000","last_undegraded":"2024-04-25T15:04:29.325092+0000","last_fullsized":"2024-04-25T15:04:29.325092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.957717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217524+0000","last_change":"2024-04-25T15:02:00.941277+0000","last_active":"2024-04-25T15:04:29.217524+0000","last_peered":"2024-04-25T15:04:29.217524+0000","last_clean":"2024-04-25T15:04:29.217524+0000","last_became_active":"2024-04-25T15:02:00.940914+0000","last_became_peered":"2024-04-25T15:02:00.940914+0000","last_unstale":"2024-04-25T15:04:29.217524+0000","last_undegraded":"2024-04-25T15:04:29.217524+0000","last_fullsized":"2024-04-25T15:04:29.217524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:00.466086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354547+0000","last_change":"2024-04-25T15:02:00.947796+0000","last_active":"2024-04-25T15:04:29.354547+0000","last_peered":"2024-04-25T15:04:29.354547+0000","last_clean":"2024-04-25T15:04:29.354547+0000","last_became_active":"2024-04-25T15:02:00.946868+0000","last_became_peered":"2024-04-25T15:02:00.946868+0000","last_unstale":"2024-04-25T15:04:29.354547+0000","last_undegraded":"2024-04-25T15:04:29.354547+0000","last_fullsized":"2024-04-25T15:04:29.354547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.825136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.078119+0000","last_change":"2024-04-25T15:02:00.943137+0000","last_active":"2024-04-25T15:04:29.078119+0000","last_peered":"2024-04-25T15:04:29.078119+0000","last_clean":"2024-04-25T15:04:29.078119+0000","last_became_active":"2024-04-25T15:02:00.942694+0000","last_became_peered":"2024-04-25T15:02:00.942694+0000","last_unstale":"2024-04-25T15:04:29.078119+0000","last_undegraded":"2024-04-25T15:04:29.078119+0000","last_fullsized":"2024-04-25T15:04:29.078119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:45.619262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642134+0000","last_change":"2024-04-25T15:02:00.946882+0000","last_active":"2024-04-25T15:04:29.642134+0000","last_peered":"2024-04-25T15:04:29.642134+0000","last_clean":"2024-04-25T15:04:29.642134+0000","last_became_active":"2024-04-25T15:02:00.945439+0000","last_became_peered":"2024-04-25T15:02:00.945439+0000","last_unstale":"2024-04-25T15:04:29.642134+0000","last_undegraded":"2024-04-25T15:04:29.642134+0000","last_fullsized":"2024-04-25T15:04:29.642134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:53.408769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507345+0000","last_change":"2024-04-25T15:02:00.943550+0000","last_active":"2024-04-25T15:04:29.507345+0000","last_peered":"2024-04-25T15:04:29.507345+0000","last_clean":"2024-04-25T15:04:29.507345+0000","last_became_active":"2024-04-25T15:02:00.942956+0000","last_became_peered":"2024-04-25T15:02:00.942956+0000","last_unstale":"2024-04-25T15:04:29.507345+0000","last_undegraded":"2024-04-25T15:04:29.507345+0000","last_fullsized":"2024-04-25T15:04:29.507345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:58.751560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507098+0000","last_change":"2024-04-25T15:02:00.949703+0000","last_active":"2024-04-25T15:04:29.507098+0000","last_peered":"2024-04-25T15:04:29.507098+0000","last_clean":"2024-04-25T15:04:29.507098+0000","last_became_active":"2024-04-25T15:02:00.948115+0000","last_became_peered":"2024-04-25T15:02:00.948115+0000","last_unstale":"2024-04-25T15:04:29.507098+0000","last_undegraded":"2024-04-25T15:04:29.507098+0000","last_fullsized":"2024-04-25T15:04:29.507098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:21.085897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324603+0000","last_change":"2024-04-25T15:02:00.955196+0000","last_active":"2024-04-25T15:04:29.324603+0000","last_peered":"2024-04-25T15:04:29.324603+0000","last_clean":"2024-04-25T15:04:29.324603+0000","last_became_active":"2024-04-25T15:02:00.954898+0000","last_became_peered":"2024-04-25T15:02:00.954898+0000","last_unstale":"2024-04-25T15:04:29.324603+0000","last_undegraded":"2024-04-25T15:04:29.324603+0000","last_fullsized":"2024-04-25T15:04:29.324603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:07.103918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642011+0000","last_change":"2024-04-25T15:02:00.947474+0000","last_active":"2024-04-25T15:04:29.642011+0000","last_peered":"2024-04-25T15:04:29.642011+0000","last_clean":"2024-04-25T15:04:29.642011+0000","last_became_active":"2024-04-25T15:02:00.947033+0000","last_became_peered":"2024-04-25T15:02:00.947033+0000","last_unstale":"2024-04-25T15:04:29.642011+0000","last_undegraded":"2024-04-25T15:04:29.642011+0000","last_fullsized":"2024-04-25T15:04:29.642011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:23.536146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083738+0000","last_change":"2024-04-25T15:02:00.948875+0000","last_active":"2024-04-25T15:04:29.083738+0000","last_peered":"2024-04-25T15:04:29.083738+0000","last_clean":"2024-04-25T15:04:29.083738+0000","last_became_active":"2024-04-25T15:02:00.947461+0000","last_became_peered":"2024-04-25T15:02:00.947461+0000","last_unstale":"2024-04-25T15:04:29.083738+0000","last_undegraded":"2024-04-25T15:04:29.083738+0000","last_fullsized":"2024-04-25T15:04:29.083738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.777173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079371+0000","last_change":"2024-04-25T15:02:00.958818+0000","last_active":"2024-04-25T15:04:29.079371+0000","last_peered":"2024-04-25T15:04:29.079371+0000","last_clean":"2024-04-25T15:04:29.079371+0000","last_became_active":"2024-04-25T15:02:00.958473+0000","last_became_peered":"2024-04-25T15:02:00.958473+0000","last_unstale":"2024-04-25T15:04:29.079371+0000","last_undegraded":"2024-04-25T15:04:29.079371+0000","last_fullsized":"2024-04-25T15:04:29.079371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:50.100287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.085077+0000","last_change":"2024-04-25T15:02:00.042644+0000","last_active":"2024-04-25T15:04:29.085077+0000","last_peered":"2024-04-25T15:04:29.085077+0000","last_clean":"2024-04-25T15:04:29.085077+0000","last_became_active":"2024-04-25T15:02:00.036346+0000","last_became_peered":"2024-04-25T15:02:00.036346+0000","last_unstale":"2024-04-25T15:04:29.085077+0000","last_undegraded":"2024-04-25T15:04:29.085077+0000","last_fullsized":"2024-04-25T15:04:29.085077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.510079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205959+0000","last_change":"2024-04-25T15:02:00.959257+0000","last_active":"2024-04-25T15:04:29.205959+0000","last_peered":"2024-04-25T15:04:29.205959+0000","last_clean":"2024-04-25T15:04:29.205959+0000","last_became_active":"2024-04-25T15:02:00.957242+0000","last_became_peered":"2024-04-25T15:02:00.957242+0000","last_unstale":"2024-04-25T15:04:29.205959+0000","last_undegraded":"2024-04-25T15:04:29.205959+0000","last_fullsized":"2024-04-25T15:04:29.205959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:44.910673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206088+0000","last_change":"2024-04-25T15:02:00.058796+0000","last_active":"2024-04-25T15:04:29.206088+0000","last_peered":"2024-04-25T15:04:29.206088+0000","last_clean":"2024-04-25T15:04:29.206088+0000","last_became_active":"2024-04-25T15:02:00.058391+0000","last_became_peered":"2024-04-25T15:02:00.058391+0000","last_unstale":"2024-04-25T15:04:29.206088+0000","last_undegraded":"2024-04-25T15:04:29.206088+0000","last_fullsized":"2024-04-25T15:04:29.206088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:08.418513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507821+0000","last_change":"2024-04-25T15:02:00.953579+0000","last_active":"2024-04-25T15:04:29.507821+0000","last_peered":"2024-04-25T15:04:29.507821+0000","last_clean":"2024-04-25T15:04:29.507821+0000","last_became_active":"2024-04-25T15:02:00.952599+0000","last_became_peered":"2024-04-25T15:02:00.952599+0000","last_unstale":"2024-04-25T15:04:29.507821+0000","last_undegraded":"2024-04-25T15:04:29.507821+0000","last_fullsized":"2024-04-25T15:04:29.507821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:33.156545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205973+0000","last_change":"2024-04-25T15:02:57.476182+0000","last_active":"2024-04-25T15:04:29.205973+0000","last_peered":"2024-04-25T15:04:29.205973+0000","last_clean":"2024-04-25T15:04:29.205973+0000","last_became_active":"2024-04-25T15:02:57.472985+0000","last_became_peered":"2024-04-25T15:02:57.472985+0000","last_unstale":"2024-04-25T15:04:29.205973+0000","last_undegraded":"2024-04-25T15:04:29.205973+0000","last_fullsized":"2024-04-25T15:04:29.205973+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.669296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325105+0000","last_change":"2024-04-25T15:02:00.946138+0000","last_active":"2024-04-25T15:04:29.325105+0000","last_peered":"2024-04-25T15:04:29.325105+0000","last_clean":"2024-04-25T15:04:29.325105+0000","last_became_active":"2024-04-25T15:02:00.945774+0000","last_became_peered":"2024-04-25T15:02:00.945774+0000","last_unstale":"2024-04-25T15:04:29.325105+0000","last_undegraded":"2024-04-25T15:04:29.325105+0000","last_fullsized":"2024-04-25T15:04:29.325105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.276821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205852+0000","last_change":"2024-04-25T15:02:00.035540+0000","last_active":"2024-04-25T15:04:29.205852+0000","last_peered":"2024-04-25T15:04:29.205852+0000","last_clean":"2024-04-25T15:04:29.205852+0000","last_became_active":"2024-04-25T15:02:00.035031+0000","last_became_peered":"2024-04-25T15:02:00.035031+0000","last_unstale":"2024-04-25T15:04:29.205852+0000","last_undegraded":"2024-04-25T15:04:29.205852+0000","last_fullsized":"2024-04-25T15:04:29.205852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:44.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213970+0000","last_change":"2024-04-25T15:02:57.471043+0000","last_active":"2024-04-25T15:04:29.213970+0000","last_peered":"2024-04-25T15:04:29.213970+0000","last_clean":"2024-04-25T15:04:29.213970+0000","last_became_active":"2024-04-25T15:02:57.470670+0000","last_became_peered":"2024-04-25T15:02:57.470670+0000","last_unstale":"2024-04-25T15:04:29.213970+0000","last_undegraded":"2024-04-25T15:04:29.213970+0000","last_fullsized":"2024-04-25T15:04:29.213970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.243201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213843+0000","last_change":"2024-04-25T15:02:00.037244+0000","last_active":"2024-04-25T15:04:29.213843+0000","last_peered":"2024-04-25T15:04:29.213843+0000","last_clean":"2024-04-25T15:04:29.213843+0000","last_became_active":"2024-04-25T15:02:00.036863+0000","last_became_peered":"2024-04-25T15:02:00.036863+0000","last_unstale":"2024-04-25T15:04:29.213843+0000","last_undegraded":"2024-04-25T15:04:29.213843+0000","last_fullsized":"2024-04-25T15:04:29.213843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:56.599999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205929+0000","last_change":"2024-04-25T15:02:00.949114+0000","last_active":"2024-04-25T15:04:29.205929+0000","last_peered":"2024-04-25T15:04:29.205929+0000","last_clean":"2024-04-25T15:04:29.205929+0000","last_became_active":"2024-04-25T15:02:00.948729+0000","last_became_peered":"2024-04-25T15:02:00.948729+0000","last_unstale":"2024-04-25T15:04:29.205929+0000","last_undegraded":"2024-04-25T15:04:29.205929+0000","last_fullsized":"2024-04-25T15:04:29.205929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.198118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217119+0000","last_change":"2024-04-25T15:02:00.045693+0000","last_active":"2024-04-25T15:04:29.217119+0000","last_peered":"2024-04-25T15:04:29.217119+0000","last_clean":"2024-04-25T15:04:29.217119+0000","last_became_active":"2024-04-25T15:02:00.045300+0000","last_became_peered":"2024-04-25T15:02:00.045300+0000","last_unstale":"2024-04-25T15:04:29.217119+0000","last_undegraded":"2024-04-25T15:04:29.217119+0000","last_fullsized":"2024-04-25T15:04:29.217119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:30.864866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213785+0000","last_change":"2024-04-25T15:02:00.964501+0000","last_active":"2024-04-25T15:04:29.213785+0000","last_peered":"2024-04-25T15:04:29.213785+0000","last_clean":"2024-04-25T15:04:29.213785+0000","last_became_active":"2024-04-25T15:02:00.964054+0000","last_became_peered":"2024-04-25T15:02:00.964054+0000","last_unstale":"2024-04-25T15:04:29.213785+0000","last_undegraded":"2024-04-25T15:04:29.213785+0000","last_fullsized":"2024-04-25T15:04:29.213785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:45.175888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642227+0000","last_change":"2024-04-25T15:02:00.050576+0000","last_active":"2024-04-25T15:04:29.642227+0000","last_peered":"2024-04-25T15:04:29.642227+0000","last_clean":"2024-04-25T15:04:29.642227+0000","last_became_active":"2024-04-25T15:02:00.050150+0000","last_became_peered":"2024-04-25T15:02:00.050150+0000","last_unstale":"2024-04-25T15:04:29.642227+0000","last_undegraded":"2024-04-25T15:04:29.642227+0000","last_fullsized":"2024-04-25T15:04:29.642227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:26.679293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205603+0000","last_change":"2024-04-25T15:02:00.959904+0000","last_active":"2024-04-25T15:04:29.205603+0000","last_peered":"2024-04-25T15:04:29.205603+0000","last_clean":"2024-04-25T15:04:29.205603+0000","last_became_active":"2024-04-25T15:02:00.957484+0000","last_became_peered":"2024-04-25T15:02:00.957484+0000","last_unstale":"2024-04-25T15:04:29.205603+0000","last_undegraded":"2024-04-25T15:04:29.205603+0000","last_fullsized":"2024-04-25T15:04:29.205603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:19.563193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213605+0000","last_change":"2024-04-25T15:02:00.037492+0000","last_active":"2024-04-25T15:04:29.213605+0000","last_peered":"2024-04-25T15:04:29.213605+0000","last_clean":"2024-04-25T15:04:29.213605+0000","last_became_active":"2024-04-25T15:02:00.037148+0000","last_became_peered":"2024-04-25T15:02:00.037148+0000","last_unstale":"2024-04-25T15:04:29.213605+0000","last_undegraded":"2024-04-25T15:04:29.213605+0000","last_fullsized":"2024-04-25T15:04:29.213605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:26.027196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083091+0000","last_change":"2024-04-25T15:02:00.959489+0000","last_active":"2024-04-25T15:04:29.083091+0000","last_peered":"2024-04-25T15:04:29.083091+0000","last_clean":"2024-04-25T15:04:29.083091+0000","last_became_active":"2024-04-25T15:02:00.958259+0000","last_became_peered":"2024-04-25T15:02:00.958259+0000","last_unstale":"2024-04-25T15:04:29.083091+0000","last_undegraded":"2024-04-25T15:04:29.083091+0000","last_fullsized":"2024-04-25T15:04:29.083091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:37.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217382+0000","last_change":"2024-04-25T15:02:00.043873+0000","last_active":"2024-04-25T15:04:29.217382+0000","last_peered":"2024-04-25T15:04:29.217382+0000","last_clean":"2024-04-25T15:04:29.217382+0000","last_became_active":"2024-04-25T15:02:00.043558+0000","last_became_peered":"2024-04-25T15:02:00.043558+0000","last_unstale":"2024-04-25T15:04:29.217382+0000","last_undegraded":"2024-04-25T15:04:29.217382+0000","last_fullsized":"2024-04-25T15:04:29.217382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:49.600388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325069+0000","last_change":"2024-04-25T15:02:00.949642+0000","last_active":"2024-04-25T15:04:29.325069+0000","last_peered":"2024-04-25T15:04:29.325069+0000","last_clean":"2024-04-25T15:04:29.325069+0000","last_became_active":"2024-04-25T15:02:00.949018+0000","last_became_peered":"2024-04-25T15:02:00.949018+0000","last_unstale":"2024-04-25T15:04:29.325069+0000","last_undegraded":"2024-04-25T15:04:29.325069+0000","last_fullsized":"2024-04-25T15:04:29.325069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:02.637921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354452+0000","last_change":"2024-04-25T15:02:00.038865+0000","last_active":"2024-04-25T15:04:29.354452+0000","last_peered":"2024-04-25T15:04:29.354452+0000","last_clean":"2024-04-25T15:04:29.354452+0000","last_became_active":"2024-04-25T15:02:00.038182+0000","last_became_peered":"2024-04-25T15:02:00.038182+0000","last_unstale":"2024-04-25T15:04:29.354452+0000","last_undegraded":"2024-04-25T15:04:29.354452+0000","last_fullsized":"2024-04-25T15:04:29.354452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:46.041917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507878+0000","last_change":"2024-04-25T15:02:00.942056+0000","last_active":"2024-04-25T15:04:29.507878+0000","last_peered":"2024-04-25T15:04:29.507878+0000","last_clean":"2024-04-25T15:04:29.507878+0000","last_became_active":"2024-04-25T15:02:00.941495+0000","last_became_peered":"2024-04-25T15:02:00.941495+0000","last_unstale":"2024-04-25T15:04:29.507878+0000","last_undegraded":"2024-04-25T15:04:29.507878+0000","last_fullsized":"2024-04-25T15:04:29.507878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:22.117055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.214112+0000","last_change":"2024-04-25T15:02:00.058087+0000","last_active":"2024-04-25T15:04:29.214112+0000","last_peered":"2024-04-25T15:04:29.214112+0000","last_clean":"2024-04-25T15:04:29.214112+0000","last_became_active":"2024-04-25T15:02:00.057681+0000","last_became_peered":"2024-04-25T15:02:00.057681+0000","last_unstale":"2024-04-25T15:04:29.214112+0000","last_undegraded":"2024-04-25T15:04:29.214112+0000","last_fullsized":"2024-04-25T15:04:29.214112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:00.445347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.325015+0000","last_change":"2024-04-25T15:02:00.945116+0000","last_active":"2024-04-25T15:04:29.325015+0000","last_peered":"2024-04-25T15:04:29.325015+0000","last_clean":"2024-04-25T15:04:29.325015+0000","last_became_active":"2024-04-25T15:02:00.944711+0000","last_became_peered":"2024-04-25T15:02:00.944711+0000","last_unstale":"2024-04-25T15:04:29.325015+0000","last_undegraded":"2024-04-25T15:04:29.325015+0000","last_fullsized":"2024-04-25T15:04:29.325015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.696456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":381,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:39.638060+0000","last_change":"2024-04-25T15:02:00.040111+0000","last_active":"2024-04-25T15:04:39.638060+0000","last_peered":"2024-04-25T15:04:39.638060+0000","last_clean":"2024-04-25T15:04:39.638060+0000","last_became_active":"2024-04-25T15:02:00.039742+0000","last_became_peered":"2024-04-25T15:02:00.039742+0000","last_unstale":"2024-04-25T15:04:39.638060+0000","last_undegraded":"2024-04-25T15:04:39.638060+0000","last_fullsized":"2024-04-25T15:04:39.638060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.456267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67246,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":228,"num_write":35,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.943035+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.942679+0000","last_became_peered":"2024-04-25T15:02:00.942679+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:19.823086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079551+0000","last_change":"2024-04-25T15:02:00.037973+0000","last_active":"2024-04-25T15:04:29.079551+0000","last_peered":"2024-04-25T15:04:29.079551+0000","last_clean":"2024-04-25T15:04:29.079551+0000","last_became_active":"2024-04-25T15:02:00.037454+0000","last_became_peered":"2024-04-25T15:02:00.037454+0000","last_unstale":"2024-04-25T15:04:29.079551+0000","last_undegraded":"2024-04-25T15:04:29.079551+0000","last_fullsized":"2024-04-25T15:04:29.079551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:15.707309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641849+0000","last_change":"2024-04-25T15:02:00.962981+0000","last_active":"2024-04-25T15:04:29.641849+0000","last_peered":"2024-04-25T15:04:29.641849+0000","last_clean":"2024-04-25T15:04:29.641849+0000","last_became_active":"2024-04-25T15:02:00.962367+0000","last_became_peered":"2024-04-25T15:02:00.962367+0000","last_unstale":"2024-04-25T15:04:29.641849+0000","last_undegraded":"2024-04-25T15:04:29.641849+0000","last_fullsized":"2024-04-25T15:04:29.641849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213964+0000","last_change":"2024-04-25T15:02:00.038161+0000","last_active":"2024-04-25T15:04:29.213964+0000","last_peered":"2024-04-25T15:04:29.213964+0000","last_clean":"2024-04-25T15:04:29.213964+0000","last_became_active":"2024-04-25T15:02:00.037667+0000","last_became_peered":"2024-04-25T15:02:00.037667+0000","last_unstale":"2024-04-25T15:04:29.213964+0000","last_undegraded":"2024-04-25T15:04:29.213964+0000","last_fullsized":"2024-04-25T15:04:29.213964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:35.225975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354351+0000","last_change":"2024-04-25T15:02:00.938458+0000","last_active":"2024-04-25T15:04:29.354351+0000","last_peered":"2024-04-25T15:04:29.354351+0000","last_clean":"2024-04-25T15:04:29.354351+0000","last_became_active":"2024-04-25T15:02:00.938112+0000","last_became_peered":"2024-04-25T15:02:00.938112+0000","last_unstale":"2024-04-25T15:04:29.354351+0000","last_undegraded":"2024-04-25T15:04:29.354351+0000","last_fullsized":"2024-04-25T15:04:29.354351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:50.939853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077852+0000","last_change":"2024-04-25T15:02:00.036437+0000","last_active":"2024-04-25T15:04:29.077852+0000","last_peered":"2024-04-25T15:04:29.077852+0000","last_clean":"2024-04-25T15:04:29.077852+0000","last_became_active":"2024-04-25T15:02:00.036067+0000","last_became_peered":"2024-04-25T15:02:00.036067+0000","last_unstale":"2024-04-25T15:04:29.077852+0000","last_undegraded":"2024-04-25T15:04:29.077852+0000","last_fullsized":"2024-04-25T15:04:29.077852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:04.298663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354095+0000","last_change":"2024-04-25T15:02:00.937747+0000","last_active":"2024-04-25T15:04:29.354095+0000","last_peered":"2024-04-25T15:04:29.354095+0000","last_clean":"2024-04-25T15:04:29.354095+0000","last_became_active":"2024-04-25T15:02:00.936617+0000","last_became_peered":"2024-04-25T15:02:00.936617+0000","last_unstale":"2024-04-25T15:04:29.354095+0000","last_undegraded":"2024-04-25T15:04:29.354095+0000","last_fullsized":"2024-04-25T15:04:29.354095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:45.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205561+0000","last_change":"2024-04-25T15:02:00.056270+0000","last_active":"2024-04-25T15:04:29.205561+0000","last_peered":"2024-04-25T15:04:29.205561+0000","last_clean":"2024-04-25T15:04:29.205561+0000","last_became_active":"2024-04-25T15:02:00.055900+0000","last_became_peered":"2024-04-25T15:02:00.055900+0000","last_unstale":"2024-04-25T15:04:29.205561+0000","last_undegraded":"2024-04-25T15:04:29.205561+0000","last_fullsized":"2024-04-25T15:04:29.205561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:12.346553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213715+0000","last_change":"2024-04-25T15:02:00.945573+0000","last_active":"2024-04-25T15:04:29.213715+0000","last_peered":"2024-04-25T15:04:29.213715+0000","last_clean":"2024-04-25T15:04:29.213715+0000","last_became_active":"2024-04-25T15:02:00.944928+0000","last_became_peered":"2024-04-25T15:02:00.944928+0000","last_unstale":"2024-04-25T15:04:29.213715+0000","last_undegraded":"2024-04-25T15:04:29.213715+0000","last_fullsized":"2024-04-25T15:04:29.213715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.830085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641863+0000","last_change":"2024-04-25T15:02:57.473783+0000","last_active":"2024-04-25T15:04:29.641863+0000","last_peered":"2024-04-25T15:04:29.641863+0000","last_clean":"2024-04-25T15:04:29.641863+0000","last_became_active":"2024-04-25T15:02:57.468466+0000","last_became_peered":"2024-04-25T15:02:57.468466+0000","last_unstale":"2024-04-25T15:04:29.641863+0000","last_undegraded":"2024-04-25T15:04:29.641863+0000","last_fullsized":"2024-04-25T15:04:29.641863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:41.182777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641910+0000","last_change":"2024-04-25T15:02:00.942284+0000","last_active":"2024-04-25T15:04:29.641910+0000","last_peered":"2024-04-25T15:04:29.641910+0000","last_clean":"2024-04-25T15:04:29.641910+0000","last_became_active":"2024-04-25T15:02:00.941859+0000","last_became_peered":"2024-04-25T15:02:00.941859+0000","last_unstale":"2024-04-25T15:04:29.641910+0000","last_undegraded":"2024-04-25T15:04:29.641910+0000","last_fullsized":"2024-04-25T15:04:29.641910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:57.976648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.083393+0000","last_change":"2024-04-25T15:02:00.061141+0000","last_active":"2024-04-25T15:04:29.083393+0000","last_peered":"2024-04-25T15:04:29.083393+0000","last_clean":"2024-04-25T15:04:29.083393+0000","last_became_active":"2024-04-25T15:02:00.059862+0000","last_became_peered":"2024-04-25T15:02:00.059862+0000","last_unstale":"2024-04-25T15:04:29.083393+0000","last_undegraded":"2024-04-25T15:04:29.083393+0000","last_fullsized":"2024-04-25T15:04:29.083393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:56.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217387+0000","last_change":"2024-04-25T15:02:00.960821+0000","last_active":"2024-04-25T15:04:29.217387+0000","last_peered":"2024-04-25T15:04:29.217387+0000","last_clean":"2024-04-25T15:04:29.217387+0000","last_became_active":"2024-04-25T15:02:00.960519+0000","last_became_peered":"2024-04-25T15:02:00.960519+0000","last_unstale":"2024-04-25T15:04:29.217387+0000","last_undegraded":"2024-04-25T15:04:29.217387+0000","last_fullsized":"2024-04-25T15:04:29.217387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354189+0000","last_change":"2024-04-25T15:02:00.047344+0000","last_active":"2024-04-25T15:04:29.354189+0000","last_peered":"2024-04-25T15:04:29.354189+0000","last_clean":"2024-04-25T15:04:29.354189+0000","last_became_active":"2024-04-25T15:02:00.046830+0000","last_became_peered":"2024-04-25T15:02:00.046830+0000","last_unstale":"2024-04-25T15:04:29.354189+0000","last_undegraded":"2024-04-25T15:04:29.354189+0000","last_fullsized":"2024-04-25T15:04:29.354189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:57.057886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.217231+0000","last_change":"2024-04-25T15:02:00.940234+0000","last_active":"2024-04-25T15:04:29.217231+0000","last_peered":"2024-04-25T15:04:29.217231+0000","last_clean":"2024-04-25T15:04:29.217231+0000","last_became_active":"2024-04-25T15:02:00.939895+0000","last_became_peered":"2024-04-25T15:02:00.939895+0000","last_unstale":"2024-04-25T15:04:29.217231+0000","last_undegraded":"2024-04-25T15:04:29.217231+0000","last_fullsized":"2024-04-25T15:04:29.217231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:11.229435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213699+0000","last_change":"2024-04-25T15:02:00.936917+0000","last_active":"2024-04-25T15:04:29.213699+0000","last_peered":"2024-04-25T15:04:29.213699+0000","last_clean":"2024-04-25T15:04:29.213699+0000","last_became_active":"2024-04-25T15:02:00.936499+0000","last_became_peered":"2024-04-25T15:02:00.936499+0000","last_unstale":"2024-04-25T15:04:29.213699+0000","last_undegraded":"2024-04-25T15:04:29.213699+0000","last_fullsized":"2024-04-25T15:04:29.213699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:20.036940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641750+0000","last_change":"2024-04-25T15:02:00.058639+0000","last_active":"2024-04-25T15:04:29.641750+0000","last_peered":"2024-04-25T15:04:29.641750+0000","last_clean":"2024-04-25T15:04:29.641750+0000","last_became_active":"2024-04-25T15:02:00.058220+0000","last_became_peered":"2024-04-25T15:02:00.058220+0000","last_unstale":"2024-04-25T15:04:29.641750+0000","last_undegraded":"2024-04-25T15:04:29.641750+0000","last_fullsized":"2024-04-25T15:04:29.641750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:59.064785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205739+0000","last_change":"2024-04-25T15:02:00.963347+0000","last_active":"2024-04-25T15:04:29.205739+0000","last_peered":"2024-04-25T15:04:29.205739+0000","last_clean":"2024-04-25T15:04:29.205739+0000","last_became_active":"2024-04-25T15:02:00.960420+0000","last_became_peered":"2024-04-25T15:02:00.960420+0000","last_unstale":"2024-04-25T15:04:29.205739+0000","last_undegraded":"2024-04-25T15:04:29.205739+0000","last_fullsized":"2024-04-25T15:04:29.205739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:46.188905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077898+0000","last_change":"2024-04-25T15:02:00.035485+0000","last_active":"2024-04-25T15:04:29.077898+0000","last_peered":"2024-04-25T15:04:29.077898+0000","last_clean":"2024-04-25T15:04:29.077898+0000","last_became_active":"2024-04-25T15:02:00.035129+0000","last_became_peered":"2024-04-25T15:02:00.035129+0000","last_unstale":"2024-04-25T15:04:29.077898+0000","last_undegraded":"2024-04-25T15:04:29.077898+0000","last_fullsized":"2024-04-25T15:04:29.077898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:06.563530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":401,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.227382+0000","last_change":"2024-04-25T15:00:40.306377+0000","last_active":"2024-04-25T15:04:31.227382+0000","last_peered":"2024-04-25T15:04:31.227382+0000","last_clean":"2024-04-25T15:04:31.227382+0000","last_became_active":"2024-04-25T15:00:40.305847+0000","last_became_peered":"2024-04-25T15:00:40.305847+0000","last_unstale":"2024-04-25T15:04:31.227382+0000","last_undegraded":"2024-04-25T15:04:31.227382+0000","last_fullsized":"2024-04-25T15:04:31.227382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:56:44.926011+0000","last_clean_scrub_stamp":"2024-04-25T14:56:44.926011+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:30.642058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641602+0000","last_change":"2024-04-25T15:02:57.474003+0000","last_active":"2024-04-25T15:04:29.641602+0000","last_peered":"2024-04-25T15:04:29.641602+0000","last_clean":"2024-04-25T15:04:29.641602+0000","last_became_active":"2024-04-25T15:02:57.473443+0000","last_became_peered":"2024-04-25T15:02:57.473443+0000","last_unstale":"2024-04-25T15:04:29.641602+0000","last_undegraded":"2024-04-25T15:04:29.641602+0000","last_fullsized":"2024-04-25T15:04:29.641602+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:30.170675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354286+0000","last_change":"2024-04-25T15:02:00.042022+0000","last_active":"2024-04-25T15:04:29.354286+0000","last_peered":"2024-04-25T15:04:29.354286+0000","last_clean":"2024-04-25T15:04:29.354286+0000","last_became_active":"2024-04-25T15:02:00.041687+0000","last_became_peered":"2024-04-25T15:02:00.041687+0000","last_unstale":"2024-04-25T15:04:29.354286+0000","last_undegraded":"2024-04-25T15:04:29.354286+0000","last_fullsized":"2024-04-25T15:04:29.354286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:36.174099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641817+0000","last_change":"2024-04-25T15:02:57.529669+0000","last_active":"2024-04-25T15:04:29.641817+0000","last_peered":"2024-04-25T15:04:29.641817+0000","last_clean":"2024-04-25T15:04:29.641817+0000","last_became_active":"2024-04-25T15:02:57.529390+0000","last_became_peered":"2024-04-25T15:02:57.529390+0000","last_unstale":"2024-04-25T15:04:29.641817+0000","last_undegraded":"2024-04-25T15:04:29.641817+0000","last_fullsized":"2024-04-25T15:04:29.641817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:36.255304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642335+0000","last_change":"2024-04-25T15:02:00.953352+0000","last_active":"2024-04-25T15:04:29.642335+0000","last_peered":"2024-04-25T15:04:29.642335+0000","last_clean":"2024-04-25T15:04:29.642335+0000","last_became_active":"2024-04-25T15:02:00.950529+0000","last_became_peered":"2024-04-25T15:02:00.950529+0000","last_unstale":"2024-04-25T15:04:29.642335+0000","last_undegraded":"2024-04-25T15:04:29.642335+0000","last_fullsized":"2024-04-25T15:04:29.642335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'98","reported_seq":212,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.812284+0000","last_change":"2024-04-25T15:02:00.040126+0000","last_active":"2024-04-25T15:04:31.812284+0000","last_peered":"2024-04-25T15:04:31.812284+0000","last_clean":"2024-04-25T15:04:31.812284+0000","last_became_active":"2024-04-25T15:02:00.036837+0000","last_became_peered":"2024-04-25T15:02:00.036837+0000","last_unstale":"2024-04-25T15:04:31.812284+0000","last_undegraded":"2024-04-25T15:04:31.812284+0000","last_fullsized":"2024-04-25T15:04:31.812284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:49.994744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":180,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206049+0000","last_change":"2024-04-25T15:02:00.962934+0000","last_active":"2024-04-25T15:04:29.206049+0000","last_peered":"2024-04-25T15:04:29.206049+0000","last_clean":"2024-04-25T15:04:29.206049+0000","last_became_active":"2024-04-25T15:02:00.960322+0000","last_became_peered":"2024-04-25T15:02:00.960322+0000","last_unstale":"2024-04-25T15:04:29.206049+0000","last_undegraded":"2024-04-25T15:04:29.206049+0000","last_fullsized":"2024-04-25T15:04:29.206049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:16.346869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.077925+0000","last_change":"2024-04-25T15:02:00.047741+0000","last_active":"2024-04-25T15:04:29.077925+0000","last_peered":"2024-04-25T15:04:29.077925+0000","last_clean":"2024-04-25T15:04:29.077925+0000","last_became_active":"2024-04-25T15:02:00.047247+0000","last_became_peered":"2024-04-25T15:02:00.047247+0000","last_unstale":"2024-04-25T15:04:29.077925+0000","last_undegraded":"2024-04-25T15:04:29.077925+0000","last_fullsized":"2024-04-25T15:04:29.077925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:26.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507811+0000","last_change":"2024-04-25T15:02:00.959149+0000","last_active":"2024-04-25T15:04:29.507811+0000","last_peered":"2024-04-25T15:04:29.507811+0000","last_clean":"2024-04-25T15:04:29.507811+0000","last_became_active":"2024-04-25T15:02:00.958698+0000","last_became_peered":"2024-04-25T15:02:00.958698+0000","last_unstale":"2024-04-25T15:04:29.507811+0000","last_undegraded":"2024-04-25T15:04:29.507811+0000","last_fullsized":"2024-04-25T15:04:29.507811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:06.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324508+0000","last_change":"2024-04-25T15:02:00.055876+0000","last_active":"2024-04-25T15:04:29.324508+0000","last_peered":"2024-04-25T15:04:29.324508+0000","last_clean":"2024-04-25T15:04:29.324508+0000","last_became_active":"2024-04-25T15:02:00.055609+0000","last_became_peered":"2024-04-25T15:02:00.055609+0000","last_unstale":"2024-04-25T15:04:29.324508+0000","last_undegraded":"2024-04-25T15:04:29.324508+0000","last_fullsized":"2024-04-25T15:04:29.324508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:07.054151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205471+0000","last_change":"2024-04-25T15:02:00.938769+0000","last_active":"2024-04-25T15:04:29.205471+0000","last_peered":"2024-04-25T15:04:29.205471+0000","last_clean":"2024-04-25T15:04:29.205471+0000","last_became_active":"2024-04-25T15:02:00.938436+0000","last_became_peered":"2024-04-25T15:02:00.938436+0000","last_unstale":"2024-04-25T15:04:29.205471+0000","last_undegraded":"2024-04-25T15:04:29.205471+0000","last_fullsized":"2024-04-25T15:04:29.205471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.206025+0000","last_change":"2024-04-25T15:02:00.031942+0000","last_active":"2024-04-25T15:04:29.206025+0000","last_peered":"2024-04-25T15:04:29.206025+0000","last_clean":"2024-04-25T15:04:29.206025+0000","last_became_active":"2024-04-25T15:02:00.030978+0000","last_became_peered":"2024-04-25T15:02:00.030978+0000","last_unstale":"2024-04-25T15:04:29.206025+0000","last_undegraded":"2024-04-25T15:04:29.206025+0000","last_fullsized":"2024-04-25T15:04:29.206025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:24.387794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641669+0000","last_change":"2024-04-25T15:02:00.939560+0000","last_active":"2024-04-25T15:04:29.641669+0000","last_peered":"2024-04-25T15:04:29.641669+0000","last_clean":"2024-04-25T15:04:29.641669+0000","last_became_active":"2024-04-25T15:02:00.939119+0000","last_became_peered":"2024-04-25T15:02:00.939119+0000","last_unstale":"2024-04-25T15:04:29.641669+0000","last_undegraded":"2024-04-25T15:04:29.641669+0000","last_fullsized":"2024-04-25T15:04:29.641669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:27.978939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079203+0000","last_change":"2024-04-25T15:02:00.046466+0000","last_active":"2024-04-25T15:04:29.079203+0000","last_peered":"2024-04-25T15:04:29.079203+0000","last_clean":"2024-04-25T15:04:29.079203+0000","last_became_active":"2024-04-25T15:02:00.046199+0000","last_became_peered":"2024-04-25T15:02:00.046199+0000","last_unstale":"2024-04-25T15:04:29.079203+0000","last_undegraded":"2024-04-25T15:04:29.079203+0000","last_fullsized":"2024-04-25T15:04:29.079203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:20.701766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205734+0000","last_change":"2024-04-25T15:02:00.965350+0000","last_active":"2024-04-25T15:04:29.205734+0000","last_peered":"2024-04-25T15:04:29.205734+0000","last_clean":"2024-04-25T15:04:29.205734+0000","last_became_active":"2024-04-25T15:02:00.963931+0000","last_became_peered":"2024-04-25T15:02:00.963931+0000","last_unstale":"2024-04-25T15:04:29.205734+0000","last_undegraded":"2024-04-25T15:04:29.205734+0000","last_fullsized":"2024-04-25T15:04:29.205734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:14.084761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205592+0000","last_change":"2024-04-25T15:02:00.041456+0000","last_active":"2024-04-25T15:04:29.205592+0000","last_peered":"2024-04-25T15:04:29.205592+0000","last_clean":"2024-04-25T15:04:29.205592+0000","last_became_active":"2024-04-25T15:02:00.040963+0000","last_became_peered":"2024-04-25T15:02:00.040963+0000","last_unstale":"2024-04-25T15:04:29.205592+0000","last_undegraded":"2024-04-25T15:04:29.205592+0000","last_fullsized":"2024-04-25T15:04:29.205592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:23.064800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354285+0000","last_change":"2024-04-25T15:02:00.944647+0000","last_active":"2024-04-25T15:04:29.354285+0000","last_peered":"2024-04-25T15:04:29.354285+0000","last_clean":"2024-04-25T15:04:29.354285+0000","last_became_active":"2024-04-25T15:02:00.944323+0000","last_became_peered":"2024-04-25T15:02:00.944323+0000","last_unstale":"2024-04-25T15:04:29.354285+0000","last_undegraded":"2024-04-25T15:04:29.354285+0000","last_fullsized":"2024-04-25T15:04:29.354285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:47.927303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324613+0000","last_change":"2024-04-25T15:02:57.529816+0000","last_active":"2024-04-25T15:04:29.324613+0000","last_peered":"2024-04-25T15:04:29.324613+0000","last_clean":"2024-04-25T15:04:29.324613+0000","last_became_active":"2024-04-25T15:02:57.529442+0000","last_became_peered":"2024-04-25T15:02:57.529442+0000","last_unstale":"2024-04-25T15:04:29.324613+0000","last_undegraded":"2024-04-25T15:04:29.324613+0000","last_fullsized":"2024-04-25T15:04:29.324613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.143614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354562+0000","last_change":"2024-04-25T15:02:00.948025+0000","last_active":"2024-04-25T15:04:29.354562+0000","last_peered":"2024-04-25T15:04:29.354562+0000","last_clean":"2024-04-25T15:04:29.354562+0000","last_became_active":"2024-04-25T15:02:00.946596+0000","last_became_peered":"2024-04-25T15:02:00.946596+0000","last_unstale":"2024-04-25T15:04:29.354562+0000","last_undegraded":"2024-04-25T15:04:29.354562+0000","last_fullsized":"2024-04-25T15:04:29.354562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:52.730375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'65","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:31.325885+0000","last_change":"2024-04-25T15:02:00.036313+0000","last_active":"2024-04-25T15:04:31.325885+0000","last_peered":"2024-04-25T15:04:31.325885+0000","last_clean":"2024-04-25T15:04:31.325885+0000","last_became_active":"2024-04-25T15:02:00.035672+0000","last_became_peered":"2024-04-25T15:02:00.035672+0000","last_unstale":"2024-04-25T15:04:31.325885+0000","last_undegraded":"2024-04-25T15:04:31.325885+0000","last_fullsized":"2024-04-25T15:04:31.325885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:32.754300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354311+0000","last_change":"2024-04-25T15:02:00.957993+0000","last_active":"2024-04-25T15:04:29.354311+0000","last_peered":"2024-04-25T15:04:29.354311+0000","last_clean":"2024-04-25T15:04:29.354311+0000","last_became_active":"2024-04-25T15:02:00.957655+0000","last_became_peered":"2024-04-25T15:02:00.957655+0000","last_unstale":"2024-04-25T15:04:29.354311+0000","last_undegraded":"2024-04-25T15:04:29.354311+0000","last_fullsized":"2024-04-25T15:04:29.354311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:28.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642550+0000","last_change":"2024-04-25T15:02:00.049714+0000","last_active":"2024-04-25T15:04:29.642550+0000","last_peered":"2024-04-25T15:04:29.642550+0000","last_clean":"2024-04-25T15:04:29.642550+0000","last_became_active":"2024-04-25T15:02:00.047784+0000","last_became_peered":"2024-04-25T15:02:00.047784+0000","last_unstale":"2024-04-25T15:04:29.642550+0000","last_undegraded":"2024-04-25T15:04:29.642550+0000","last_fullsized":"2024-04-25T15:04:29.642550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:01:59.005645+0000","last_clean_scrub_stamp":"2024-04-25T15:01:59.005645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.817028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.354512+0000","last_change":"2024-04-25T15:02:00.942793+0000","last_active":"2024-04-25T15:04:29.354512+0000","last_peered":"2024-04-25T15:04:29.354512+0000","last_clean":"2024-04-25T15:04:29.354512+0000","last_became_active":"2024-04-25T15:02:00.942449+0000","last_became_peered":"2024-04-25T15:02:00.942449+0000","last_unstale":"2024-04-25T15:04:29.354512+0000","last_undegraded":"2024-04-25T15:04:29.354512+0000","last_fullsized":"2024-04-25T15:04:29.354512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.220738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641721+0000","last_change":"2024-04-25T15:02:00.938618+0000","last_active":"2024-04-25T15:04:29.641721+0000","last_peered":"2024-04-25T15:04:29.641721+0000","last_clean":"2024-04-25T15:04:29.641721+0000","last_became_active":"2024-04-25T15:02:00.938208+0000","last_became_peered":"2024-04-25T15:02:00.938208+0000","last_unstale":"2024-04-25T15:04:29.641721+0000","last_undegraded":"2024-04-25T15:04:29.641721+0000","last_fullsized":"2024-04-25T15:04:29.641721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.104539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079375+0000","last_change":"2024-04-25T15:02:00.939762+0000","last_active":"2024-04-25T15:04:29.079375+0000","last_peered":"2024-04-25T15:04:29.079375+0000","last_clean":"2024-04-25T15:04:29.079375+0000","last_became_active":"2024-04-25T15:02:00.938617+0000","last_became_peered":"2024-04-25T15:02:00.938617+0000","last_unstale":"2024-04-25T15:04:29.079375+0000","last_undegraded":"2024-04-25T15:04:29.079375+0000","last_fullsized":"2024-04-25T15:04:29.079375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:48.480214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079220+0000","last_change":"2024-04-25T15:02:00.958085+0000","last_active":"2024-04-25T15:04:29.079220+0000","last_peered":"2024-04-25T15:04:29.079220+0000","last_clean":"2024-04-25T15:04:29.079220+0000","last_became_active":"2024-04-25T15:02:00.957743+0000","last_became_peered":"2024-04-25T15:02:00.957743+0000","last_unstale":"2024-04-25T15:04:29.079220+0000","last_undegraded":"2024-04-25T15:04:29.079220+0000","last_fullsized":"2024-04-25T15:04:29.079220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.507354+0000","last_change":"2024-04-25T15:02:00.943725+0000","last_active":"2024-04-25T15:04:29.507354+0000","last_peered":"2024-04-25T15:04:29.507354+0000","last_clean":"2024-04-25T15:04:29.507354+0000","last_became_active":"2024-04-25T15:02:00.943187+0000","last_became_peered":"2024-04-25T15:02:00.943187+0000","last_unstale":"2024-04-25T15:04:29.507354+0000","last_undegraded":"2024-04-25T15:04:29.507354+0000","last_fullsized":"2024-04-25T15:04:29.507354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:50.330185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642445+0000","last_change":"2024-04-25T15:02:00.947672+0000","last_active":"2024-04-25T15:04:29.642445+0000","last_peered":"2024-04-25T15:04:29.642445+0000","last_clean":"2024-04-25T15:04:29.642445+0000","last_became_active":"2024-04-25T15:02:00.946866+0000","last_became_peered":"2024-04-25T15:02:00.946866+0000","last_unstale":"2024-04-25T15:04:29.642445+0000","last_undegraded":"2024-04-25T15:04:29.642445+0000","last_fullsized":"2024-04-25T15:04:29.642445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:38.976251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641969+0000","last_change":"2024-04-25T15:02:57.470460+0000","last_active":"2024-04-25T15:04:29.641969+0000","last_peered":"2024-04-25T15:04:29.641969+0000","last_clean":"2024-04-25T15:04:29.641969+0000","last_became_active":"2024-04-25T15:02:57.470013+0000","last_became_peered":"2024-04-25T15:02:57.470013+0000","last_unstale":"2024-04-25T15:04:29.641969+0000","last_undegraded":"2024-04-25T15:04:29.641969+0000","last_fullsized":"2024-04-25T15:04:29.641969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:07.351115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.079804+0000","last_change":"2024-04-25T15:02:00.946775+0000","last_active":"2024-04-25T15:04:29.079804+0000","last_peered":"2024-04-25T15:04:29.079804+0000","last_clean":"2024-04-25T15:04:29.079804+0000","last_became_active":"2024-04-25T15:02:00.946465+0000","last_became_peered":"2024-04-25T15:02:00.946465+0000","last_unstale":"2024-04-25T15:04:29.079804+0000","last_undegraded":"2024-04-25T15:04:29.079804+0000","last_fullsized":"2024-04-25T15:04:29.079804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:46.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205903+0000","last_change":"2024-04-25T15:02:00.964299+0000","last_active":"2024-04-25T15:04:29.205903+0000","last_peered":"2024-04-25T15:04:29.205903+0000","last_clean":"2024-04-25T15:04:29.205903+0000","last_became_active":"2024-04-25T15:02:00.963823+0000","last_became_peered":"2024-04-25T15:02:00.963823+0000","last_unstale":"2024-04-25T15:04:29.205903+0000","last_undegraded":"2024-04-25T15:04:29.205903+0000","last_fullsized":"2024-04-25T15:04:29.205903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:05.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.213572+0000","last_change":"2024-04-25T15:02:00.960304+0000","last_active":"2024-04-25T15:04:29.213572+0000","last_peered":"2024-04-25T15:04:29.213572+0000","last_clean":"2024-04-25T15:04:29.213572+0000","last_became_active":"2024-04-25T15:02:00.960006+0000","last_became_peered":"2024-04-25T15:02:00.960006+0000","last_unstale":"2024-04-25T15:04:29.213572+0000","last_undegraded":"2024-04-25T15:04:29.213572+0000","last_fullsized":"2024-04-25T15:04:29.213572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.877672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.205767+0000","last_change":"2024-04-25T15:02:00.961106+0000","last_active":"2024-04-25T15:04:29.205767+0000","last_peered":"2024-04-25T15:04:29.205767+0000","last_clean":"2024-04-25T15:04:29.205767+0000","last_became_active":"2024-04-25T15:02:00.960663+0000","last_became_peered":"2024-04-25T15:02:00.960663+0000","last_unstale":"2024-04-25T15:04:29.205767+0000","last_undegraded":"2024-04-25T15:04:29.205767+0000","last_fullsized":"2024-04-25T15:04:29.205767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:51.517292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.324892+0000","last_change":"2024-04-25T15:02:00.961597+0000","last_active":"2024-04-25T15:04:29.324892+0000","last_peered":"2024-04-25T15:04:29.324892+0000","last_clean":"2024-04-25T15:04:29.324892+0000","last_became_active":"2024-04-25T15:02:00.961224+0000","last_became_peered":"2024-04-25T15:02:00.961224+0000","last_unstale":"2024-04-25T15:04:29.324892+0000","last_undegraded":"2024-04-25T15:04:29.324892+0000","last_fullsized":"2024-04-25T15:04:29.324892+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:50.566241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.642103+0000","last_change":"2024-04-25T15:02:00.947941+0000","last_active":"2024-04-25T15:04:29.642103+0000","last_peered":"2024-04-25T15:04:29.642103+0000","last_clean":"2024-04-25T15:04:29.642103+0000","last_became_active":"2024-04-25T15:02:00.947505+0000","last_became_peered":"2024-04-25T15:02:00.947505+0000","last_unstale":"2024-04-25T15:04:29.642103+0000","last_undegraded":"2024-04-25T15:04:29.642103+0000","last_fullsized":"2024-04-25T15:04:29.642103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T15:04:29.641698+0000","last_change":"2024-04-25T15:02:00.942173+0000","last_active":"2024-04-25T15:04:29.641698+0000","last_peered":"2024-04-25T15:04:29.641698+0000","last_clean":"2024-04-25T15:04:29.641698+0000","last_became_active":"2024-04-25T15:02:00.941643+0000","last_became_peered":"2024-04-25T15:02:00.941643+0000","last_unstale":"2024-04-25T15:04:29.641698+0000","last_undegraded":"2024-04-25T15:04:29.641698+0000","last_fullsized":"2024-04-25T15:04:29.641698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:02:00.013667+0000","last_clean_scrub_stamp":"2024-04-25T15:02:00.013667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:31.330328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":789492,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":674,"num_read_kb":542,"num_write":328,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":2368476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285612,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29096,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716056,"statfs":{"total":95995035648,"available":95965241344,"internally_reserved":0,"allocated":2007040,"data_stored":1464906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710776,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28368,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716784,"statfs":{"total":95995035648,"available":95965986816,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972415,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1384448,"data_stored":872547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1069816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":545995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":546029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28824,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716328,"statfs":{"total":95995035648,"available":95965519872,"internally_reserved":0,"allocated":1728512,"data_stored":1200535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":478771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724623,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1929216,"data_stored":1397648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":395672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:04:44.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 -- 172.21.15.19:0/275011399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 msgr2=0x7f32b0069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 --2- 172.21.15.19:0/275011399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b0069a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f329c009900 tx=0x7f329c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 -- 172.21.15.19:0/275011399 shutdown_connections 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 --2- 172.21.15.19:0/275011399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b010e5b0 0x7f32b01109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 --2- 172.21.15.19:0/275011399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b0069a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 --2- 172.21.15.19:0/275011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b0102c50 0x7f32b00690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 -- 172.21.15.19:0/275011399 >> 172.21.15.19:0/275011399 conn(0x7f32b0076790 msgr2=0x7f32b0076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 -- 172.21.15.19:0/275011399 shutdown_connections 2024-04-25T15:04:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.327+0000 7f32afe00640 1 -- 172.21.15.19:0/275011399 wait complete. 2024-04-25T15:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.328+0000 7f32afe00640 1 Processor -- start 2024-04-25T15:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.328+0000 7f32afe00640 1 -- start start 2024-04-25T15:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b010e5b0 0x7f32b019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01141e0 con 0x7f32b0102c50 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0114060 con 0x7f32b010e5b0 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32afe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0114360 con 0x7f32b0069620 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32ae000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32ae000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55300/0 (socket says 172.21.15.19:55300) 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32ae000640 1 -- 172.21.15.19:0/413709765 learned_addr learned my addr 172.21.15.19:0/413709765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32aea00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.329+0000 7f32af400640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b010e5b0 0x7f32b019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 -- 172.21.15.19:0/413709765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 msgr2=0x7f32b01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 -- 172.21.15.19:0/413709765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b010e5b0 msgr2=0x7f32b019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b010e5b0 0x7f32b019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 -- 172.21.15.19:0/413709765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b019ad20 con 0x7f32b0102c50 2024-04-25T15:04:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32aea00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.330+0000 7f32ae000640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f329c005ab0 tx=0x7f329c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.331+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c03f070 con 0x7f32b0102c50 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.331+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c031d60 con 0x7f32b0102c50 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.331+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b019afb0 con 0x7f32b0102c50 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.331+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c035030 con 0x7f32b0102c50 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.331+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a72b0 con 0x7f32b0102c50 2024-04-25T15:04:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.332+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b0068c90 con 0x7f32b0102c50 2024-04-25T15:04:44.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.335+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f329c042070 con 0x7f32b0102c50 2024-04-25T15:04:44.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.336+0000 7f32a7400640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 0x7f32940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.336+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f329c0f1730 con 0x7f32b0102c50 2024-04-25T15:04:44.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.337+0000 7f32aea00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 0x7f32940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.337+0000 7f32aea00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 0x7f32940a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f32a00095f0 tx=0x7f32a0005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:44.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.339+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f329c0bd5e0 con 0x7f32b0102c50 2024-04-25T15:04:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.553+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32b01a74e0 con 0x7f32b0102c50 2024-04-25T15:04:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.558+0000 7f32a7400640 1 -- 172.21.15.19:0/413709765 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22954 (secure 0 0 0) 0x7f329c0c18e0 con 0x7f32b0102c50 2024-04-25T15:04:44.560 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:44.560 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:04:29.064393+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:44.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 msgr2=0x7f32940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 0x7f32940a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f32a00095f0 tx=0x7f32a0005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 msgr2=0x7f32b01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f329c005ab0 tx=0x7f329c003c80 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 shutdown_connections 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32b010e5b0 0x7f32b019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b0102c50 0x7f32b01a0930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f32940a3db0 0x7f32940a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 --2- 172.21.15.19:0/413709765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0069620 0x7f32b01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.561+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 >> 172.21.15.19:0/413709765 conn(0x7f32b0076790 msgr2=0x7f32b00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.562+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 shutdown_connections 2024-04-25T15:04:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.562+0000 7f32afe00640 1 -- 172.21.15.19:0/413709765 wait complete. 2024-04-25T15:04:44.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:04:44.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:04:44.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:44 smithi083 ceph-mon[32919]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:44.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:44 smithi083 ceph-mon[32919]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 -- 172.21.15.19:0/773794720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0104c20 msgr2=0x7f72b0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 --2- 172.21.15.19:0/773794720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0104c20 0x7f72b0108c60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7298009860 tx=0x7f729802a340 comp rx=0 tx=0).stop 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 -- 172.21.15.19:0/773794720 shutdown_connections 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 --2- 172.21.15.19:0/773794720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0109340 0x7f72b0110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 --2- 172.21.15.19:0/773794720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0104c20 0x7f72b0108c60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 --2- 172.21.15.19:0/773794720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b0104270 0x7f72b0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 -- 172.21.15.19:0/773794720 >> 172.21.15.19:0/773794720 conn(0x7f72b00ffa90 msgr2=0x7f72b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 -- 172.21.15.19:0/773794720 shutdown_connections 2024-04-25T15:04:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.794+0000 7f72b4a00640 1 -- 172.21.15.19:0/773794720 wait complete. 2024-04-25T15:04:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 Processor -- start 2024-04-25T15:04:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 -- start start 2024-04-25T15:04:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 0x7f72b01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 0x7f72b019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0114280 con 0x7f72b019a660 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0114100 con 0x7f72b0109340 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.796+0000 7f72b4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0114400 con 0x7f72b0104270 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 0x7f72b019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 0x7f72b01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44830/0 (socket says 172.21.15.19:44830) 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 0x7f72b01a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44884/0 (socket says 172.21.15.19:44884) 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 -- 172.21.15.19:0/2972838267 learned_addr learned my addr 172.21.15.19:0/2972838267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 0x7f72b019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55322/0 (socket says 172.21.15.19:55322) 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 -- 172.21.15.19:0/2972838267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 msgr2=0x7f72b01a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 0x7f72b01a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 -- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 msgr2=0x7f72b019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 0x7f72b019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 -- 172.21.15.19:0/2972838267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b019b260 con 0x7f72b0109340 2024-04-25T15:04:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.797+0000 7f72aea00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7298000c00 tx=0x7f7298032e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.798+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298040020 con 0x7f72b0109340 2024-04-25T15:04:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.798+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7298030450 con 0x7f72b0109340 2024-04-25T15:04:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.798+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298037040 con 0x7f72b0109340 2024-04-25T15:04:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.798+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b019b4f0 con 0x7f72b0109340 2024-04-25T15:04:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.798+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01a7630 con 0x7f72b0109340 2024-04-25T15:04:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.799+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0105740 con 0x7f72b0109340 2024-04-25T15:04:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.801+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7298043020 con 0x7f72b0109340 2024-04-25T15:04:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.803+0000 7f72a7e00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 0x7f72940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.803+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f72980f0670 con 0x7f72b0109340 2024-04-25T15:04:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.803+0000 7f72af400640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 0x7f72940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.804+0000 7f72af400640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 0x7f72940a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009a60 tx=0x7f72a0010040 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:44 smithi102 ceph-mon[33005]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:44 smithi102 ceph-mon[33005]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:44.807+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f729804a020 con 0x7f72b0109340 2024-04-25T15:04:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:44 smithi019 ceph-mon[28090]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:44 smithi019 ceph-mon[28090]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:04:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.015+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72b010c570 con 0x7f72b0109340 2024-04-25T15:04:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.016+0000 7f72a7e00640 1 -- 172.21.15.19:0/2972838267 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f72980bcc80 con 0x7f72b0109340 2024-04-25T15:04:45.018 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 msgr2=0x7f72940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 0x7f72940a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009a60 tx=0x7f72a0010040 comp rx=0 tx=0).stop 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 msgr2=0x7f72b01a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7298000c00 tx=0x7f7298032e10 comp rx=0 tx=0).stop 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 shutdown_connections 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b019a660 0x7f72b019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b0109340 0x7f72b01a0ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f72940a3db0 0x7f72940a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.019+0000 7f72b4a00640 1 --2- 172.21.15.19:0/2972838267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72b0104270 0x7f72b01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.020+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 >> 172.21.15.19:0/2972838267 conn(0x7f72b00ffa90 msgr2=0x7f72b0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.020+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 shutdown_connections 2024-04-25T15:04:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.020+0000 7f72b4a00640 1 -- 172.21.15.19:0/2972838267 wait complete. 2024-04-25T15:04:45.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:04:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 -- 172.21.15.19:0/1494569363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa940107270 msgr2=0x7fa94010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 --2- 172.21.15.19:0/1494569363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa940107270 0x7fa94010ee40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa938009ca0 tx=0x7fa93802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 -- 172.21.15.19:0/1494569363 shutdown_connections 2024-04-25T15:04:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 --2- 172.21.15.19:0/1494569363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa940107270 0x7fa94010ee40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 --2- 172.21.15.19:0/1494569363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940102b50 0x7fa940106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 --2- 172.21.15.19:0/1494569363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa940102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.213+0000 7fa948600640 1 -- 172.21.15.19:0/1494569363 >> 172.21.15.19:0/1494569363 conn(0x7fa9400793e0 msgr2=0x7fa9400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.214+0000 7fa948600640 1 -- 172.21.15.19:0/1494569363 shutdown_connections 2024-04-25T15:04:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.214+0000 7fa948600640 1 -- 172.21.15.19:0/1494569363 wait complete. 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 Processor -- start 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 -- start start 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa9401a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940107270 0x7fa9401a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 0x7fa94019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401121c0 con 0x7fa9401021a0 2024-04-25T15:04:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112040 con 0x7fa940107270 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.215+0000 7fa948600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112340 con 0x7fa94019a650 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa945c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa9401a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa945200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940107270 0x7fa9401a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa945c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa9401a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55348/0 (socket says 172.21.15.19:55348) 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa945c00640 1 -- 172.21.15.19:0/2683434126 learned_addr learned my addr 172.21.15.19:0/2683434126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 0x7fa94019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 -- 172.21.15.19:0/2683434126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940107270 msgr2=0x7fa9401a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940107270 0x7fa9401a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 -- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 msgr2=0x7fa9401a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa9401a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.216+0000 7fa946600640 1 -- 172.21.15.19:0/2683434126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94019b250 con 0x7fa94019a650 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa946600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 0x7fa94019aad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa93802ade0 tx=0x7fa938031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa93e000640 1 -- 172.21.15.19:0/2683434126 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938003970 con 0x7fa94019a650 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa93e000640 1 -- 172.21.15.19:0/2683434126 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa938004040 con 0x7fa94019a650 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94019b4e0 con 0x7fa94019a650 2024-04-25T15:04:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa93e000640 1 -- 172.21.15.19:0/2683434126 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93802f8c0 con 0x7fa94019a650 2024-04-25T15:04:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.217+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9401a76d0 con 0x7fa94019a650 2024-04-25T15:04:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.221+0000 7fa94b3ab740 1 Processor -- start 2024-04-25T15:04:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa93e000640 1 -- 172.21.15.19:0/2683434126 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa938003b10 con 0x7fa94019a650 2024-04-25T15:04:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 -- start start 2024-04-25T15:04:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc2820 0x560a26dc2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a26d28900 con 0x560a26dc2820 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a26cceef0 con 0x560a26dc3140 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.222+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560a26d248c0 con 0x560a26dcda00 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc2820 0x560a26dc2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44850/0 (socket says 172.21.15.19:44850) 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 -- 172.21.15.19:0/2616470174 learned_addr learned my addr 172.21.15.19:0/2616470174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945200640 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 -- 172.21.15.19:0/2616470174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 msgr2=0x560a26dcfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 -- 172.21.15.19:0/2616470174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc2820 msgr2=0x560a26dc2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc2820 0x560a26dc2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 -- 172.21.15.19:0/2616470174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a26a3b210 con 0x560a26dc3140 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa93e000640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 0x7fa9280a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa945c00640 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa934009dc0 tx=0x7fa934011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae587bb7aceee69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.223+0000 7fa93e000640 1 -- 172.21.15.19:0/2683434126 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa9380f1930 con 0x7fa94019a650 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.224+0000 7fa945200640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 0x7fa9280a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.224+0000 7fa932a00640 1 -- 172.21.15.19:0/2616470174 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934005220 con 0x560a26dc3140 2024-04-25T15:04:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.224+0000 7fa932a00640 1 -- 172.21.15.19:0/2616470174 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa934004c50 con 0x560a26dc3140 2024-04-25T15:04:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.224+0000 7fa932a00640 1 -- 172.21.15.19:0/2616470174 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934019780 con 0x560a26dc3140 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa945200640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 0x7fa9280a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x560a26dc0150 tx=0x7fa92c015040 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2616470174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 msgr2=0x560a26dcd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa934009dc0 tx=0x7fa934011810 comp rx=0 tx=0).stop 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2616470174 shutdown_connections 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc3140 0x560a26dcd4c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2616470174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc2820 0x560a26dc2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2616470174 >> 172.21.15.19:0/2616470174 conn(0x560a26dbed30 msgr2=0x560a26dbf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2616470174 shutdown_connections 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.225+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2616470174 wait complete. 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.226+0000 7fa94b3ab740 1 -- start start 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.226+0000 7fa94b3ab740 1 Processor -- start 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 0x560a26dca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dcaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a26dd0630 con 0x560a26dc3140 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a26ce2310 con 0x560a26dc2820 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560a26dd07b0 con 0x560a26dcda00 2024-04-25T15:04:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa94b3ab740 10 client.0 _ll_register_callbacks cb 0x560a26cb0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa945200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dcaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa946600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa945200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dcaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55366/0 (socket says 172.21.15.19:55366) 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa945c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 0x560a26dca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa945200640 1 -- 172.21.15.19:0/2803180677 learned_addr learned my addr 172.21.15.19:0/2803180677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa946600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44916/0 (socket says 172.21.15.19:44916) 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa946600640 1 -- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 msgr2=0x560a26dca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.227+0000 7fa946600640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 0x560a26dca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa946600640 1 -- 172.21.15.19:0/2803180677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 msgr2=0x560a26dcaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa946600640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dcaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa946600640 1 -- 172.21.15.19:0/2803180677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a26b07620 con 0x560a26dcda00 2024-04-25T15:04:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa945c00640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 0x560a26dca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa946600640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9380ef0a0 tx=0x7fa9380bd380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caed57ac1c90c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa927400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380bdf40 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa9380ef240 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa934004c50 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9380ef440 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa94b3ab740 10 client.25279 fetch_fsmap learned FSMap version 20 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.228+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a26dd0630 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa934004c50 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa9380bdb30 con 0x560a26dcda00 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 10 client.25279 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 10 client.25279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T15:04:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 20 client.25279 populate_metadata read hostname 'smithi019' 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26dddd50 0x560a26de0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 4 client.25279 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a26d3c8b0 con 0x560a26dddd50 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.229+0000 7fa945c00640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26dddd50 0x560a26de0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.230+0000 7fa945c00640 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26dddd50 0x560a26de0180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9877a1efca750750 server_cookie=e7d935de2b5dec20 in_seq=0 out_seq=0 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.230+0000 7fa927400640 10 client.25279 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa927400640 1 -- 172.21.15.19:0/2803180677 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a26d3c8b0 con 0x560a26dddd50 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa927400640 10 client.25279 handle_command_reply: tid=0 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 client.25279 shutdown 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 msgr2=0x560a26dcb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9380ef0a0 tx=0x7fa9380bd380 comp rx=0 tx=0).stop 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 shutdown_connections 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26dddd50 0x560a26de0180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dcda00 0x560a26dcb430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dcaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa927400640 0 client.25279 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/2803180677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dc2820 0x560a26dca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 >> 172.21.15.19:0/2803180677 conn(0x560a26dbed30 msgr2=0x560a26dc4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 shutdown_connections 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 1 -- 172.21.15.19:0/2803180677 wait complete. 2024-04-25T15:04:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.231+0000 7fa94b3ab740 20 client.25279 trim_cache size 0 max 16384 2024-04-25T15:04:45.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 -- 172.21.15.19:0/3676088922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1029f0 msgr2=0x7f6b1c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3676088922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1029f0 0x7f6b1c106a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009900 tx=0x7f6b0802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 -- 172.21.15.19:0/3676088922 shutdown_connections 2024-04-25T15:04:45.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3676088922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c107110 0x7f6b1c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3676088922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1029f0 0x7f6b1c106a30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3676088922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c102040 0x7f6b1c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 -- 172.21.15.19:0/3676088922 >> 172.21.15.19:0/3676088922 conn(0x7f6b1c0fd920 msgr2=0x7f6b1c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 -- 172.21.15.19:0/3676088922 shutdown_connections 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.243+0000 7f6b1be00640 1 -- 172.21.15.19:0/3676088922 wait complete. 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.244+0000 7f6b1be00640 1 Processor -- start 2024-04-25T15:04:45.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 -- start start 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c102040 0x7f6b1c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 0x7f6b1c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c112030 con 0x7f6b1c107110 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111eb0 con 0x7f6b1c102040 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c1121b0 con 0x7f6b1c19cf00 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.245+0000 7f6b1a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55380/0 (socket says 172.21.15.19:55380) 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 -- 172.21.15.19:0/3425192628 learned_addr learned my addr 172.21.15.19:0/3425192628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1b400640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 0x7f6b1c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1aa00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c102040 0x7f6b1c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 -- 172.21.15.19:0/3425192628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 msgr2=0x7f6b1c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 0x7f6b1c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 -- 172.21.15.19:0/3425192628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c102040 msgr2=0x7f6b1c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c102040 0x7f6b1c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 -- 172.21.15.19:0/3425192628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c19d9f0 con 0x7f6b1c107110 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1b400640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 0x7f6b1c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.246+0000 7f6b1a000640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006010 tx=0x7f6b08004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.247+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0803f030 con 0x7f6b1c107110 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.247+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08003aa0 con 0x7f6b1c107110 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.247+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08035070 con 0x7f6b1c107110 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.247+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c19dc80 con 0x7f6b1c107110 2024-04-25T15:04:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.247+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a53f0 con 0x7f6b1c107110 2024-04-25T15:04:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.248+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c0690e0 con 0x7f6b1c107110 2024-04-25T15:04:45.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.250+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6b08042030 con 0x7f6b1c107110 2024-04-25T15:04:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.252+0000 7fa94b3ab740 1 Processor -- start 2024-04-25T15:04:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.252+0000 7fa94b3ab740 1 -- start start 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.252+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.252+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26de5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26de5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401adc80 con 0x560a26dc3140 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401b0500 con 0x560a26dcda00 2024-04-25T15:04:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401ae2f0 con 0x560a26dc2820 2024-04-25T15:04:45.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.253+0000 7f6b13400640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 0x7f6b000a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44944/0 (socket says 172.21.15.19:44944) 2024-04-25T15:04:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 -- 172.21.15.19:0/79460897 learned_addr learned my addr 172.21.15.19:0/79460897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945200640 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26de5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.253+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6b080f09a0 con 0x7f6b1c107110 2024-04-25T15:04:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa946600640 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26de5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.253+0000 7f6b1aa00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 0x7f6b000a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 -- 172.21.15.19:0/79460897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 msgr2=0x560a26de5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26de5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 -- 172.21.15.19:0/79460897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 msgr2=0x560a26de5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26de5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.253+0000 7fa945c00640 1 -- 172.21.15.19:0/79460897 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a26dcb980 con 0x560a26dc2820 2024-04-25T15:04:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.254+0000 7fa945c00640 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa93401b040 tx=0x7fa934005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7736d89b8a064d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.254+0000 7fa932000640 1 -- 172.21.15.19:0/79460897 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93402cbc0 con 0x560a26dc2820 2024-04-25T15:04:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.254+0000 7fa932000640 1 -- 172.21.15.19:0/79460897 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9340044f0 con 0x560a26dc2820 2024-04-25T15:04:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.254+0000 7fa932000640 1 -- 172.21.15.19:0/79460897 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9340035f0 con 0x560a26dc2820 2024-04-25T15:04:45.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.254+0000 7f6b1aa00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 0x7f6b000a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009bd0 tx=0x7f6b0c005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- 172.21.15.19:0/79460897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 msgr2=0x560a26dd94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa93401b040 tx=0x7fa934005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- 172.21.15.19:0/79460897 shutdown_connections 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26de5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26de5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/79460897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd94f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- 172.21.15.19:0/79460897 >> 172.21.15.19:0/79460897 conn(0x560a26dbed30 msgr2=0x560a26dc5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- 172.21.15.19:0/79460897 shutdown_connections 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- 172.21.15.19:0/79460897 wait complete. 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.255+0000 7fa94b3ab740 1 -- start start 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 Processor -- start 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26dd3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401ae9c0 con 0x560a26dc3140 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401afe30 con 0x560a26dcda00 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9401af090 con 0x560a26dc2820 2024-04-25T15:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa94b3ab740 10 client.0 _ll_register_callbacks cb 0x7fa9380bd610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa945200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.256+0000 7fa945200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55390/0 (socket says 172.21.15.19:55390) 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa945200640 1 -- 172.21.15.19:0/217146770 learned_addr learned my addr 172.21.15.19:0/217146770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26dd3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa945c00640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 -- 172.21.15.19:0/217146770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 msgr2=0x560a26dd5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 -- 172.21.15.19:0/217146770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 msgr2=0x560a26dd3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 -- 172.21.15.19:0/217146770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a26dd3c50 con 0x560a26dcda00 2024-04-25T15:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa945200640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:45.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.257+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b080bc850 con 0x7f6b1c107110 2024-04-25T15:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa946600640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26dd3590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b9ca0 tx=0x7fa9380ba0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d906045ad016f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa927400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T15:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.257+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938009540 con 0x560a26dcda00 2024-04-25T15:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa9380bcda0 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9340044f0 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9380bdca0 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa94b3ab740 10 client.25268 fetch_fsmap learned FSMap version 20 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a26dd3c50 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9340044f0 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.258+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa9380beb70 con 0x560a26dcda00 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 10 client.25268 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 20 client.25268 populate_metadata read hostname 'smithi019' 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26e44610 0x560a26e46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 4 client.25268 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T15:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 --> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a26d36320 con 0x560a26e44610 2024-04-25T15:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.259+0000 7fa945c00640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26e44610 0x560a26e46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.260+0000 7fa945c00640 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26e44610 0x560a26e46a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d1831a9ca83227 server_cookie=bc23a1dd9e243259 in_seq=0 out_seq=0 2024-04-25T15:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.260+0000 7fa927400640 10 client.25268 ms_handle_connect on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.271+0000 7fa927400640 1 -- 172.21.15.19:0/217146770 <== mds.0 v2:172.21.15.83:6838/2949462770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a26d36320 con 0x560a26e44610 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.271+0000 7fa927400640 10 client.25268 handle_command_reply: tid=0 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.271+0000 7fa94b3ab740 1 client.25268 shutdown 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 msgr2=0x560a26dd3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26dd3590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b9ca0 tx=0x7fa9380ba0c0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 shutdown_connections 2024-04-25T15:04:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:6838/2949462770,v1:172.21.15.83:6839/2949462770] conn(0x560a26e44610 0x560a26e46a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a26dcda00 0x560a26dd3590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a26dc3140 0x560a26dd3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 --2- 172.21.15.19:0/217146770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a26dc2820 0x560a26dd5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa927400640 0 client.25268 ms_handle_reset on v2:172.21.15.83:6838/2949462770 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 >> 172.21.15.19:0/217146770 conn(0x560a26dbed30 msgr2=0x560a26dca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 shutdown_connections 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 1 -- 172.21.15.19:0/217146770 wait complete. 2024-04-25T15:04:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.272+0000 7fa94b3ab740 20 client.25268 trim_cache size 0 max 16384 2024-04-25T15:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:{ 2024-04-25T15:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "message": "", 2024-04-25T15:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout: "return_code": 0 2024-04-25T15:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stdout:} 2024-04-25T15:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 msgr2=0x7fa9280a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 0x7fa9280a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x560a26dc0150 tx=0x7fa92c015040 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 msgr2=0x7fa94019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 0x7fa94019aad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa93802ade0 tx=0x7fa938031660 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 shutdown_connections 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fa9280a3ee0 0x7fa9280a63a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa94019a650 0x7fa94019aad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa940107270 0x7fa9401a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 --2- 172.21.15.19:0/2683434126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9401021a0 0x7fa9401a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 >> 172.21.15.19:0/2683434126 conn(0x7fa9400793e0 msgr2=0x7fa9400ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 shutdown_connections 2024-04-25T15:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi019.front.sepia.ceph.com.smithi019.stderr:2024-04-25T15:04:45.273+0000 7fa948600640 1 -- 172.21.15.19:0/2683434126 wait complete. 2024-04-25T15:04:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.468+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b1c10a340 con 0x7f6b1c107110 2024-04-25T15:04:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.469+0000 7f6b13400640 1 -- 172.21.15.19:0/3425192628 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f6b080c0b50 con 0x7f6b1c107110 2024-04-25T15:04:45.470 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 msgr2=0x7f6b000a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 0x7f6b000a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009bd0 tx=0x7f6b0c005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 msgr2=0x7f6b1c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006010 tx=0x7f6b08004060 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 shutdown_connections 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b000a3e30 0x7f6b000a62f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c19cf00 0x7f6b1c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c107110 0x7f6b1c198bd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 --2- 172.21.15.19:0/3425192628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b1c102040 0x7f6b1c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 >> 172.21.15.19:0/3425192628 conn(0x7f6b1c0fd920 msgr2=0x7f6b1c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.471+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 shutdown_connections 2024-04-25T15:04:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.472+0000 7f6b1be00640 1 -- 172.21.15.19:0/3425192628 wait complete. 2024-04-25T15:04:45.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3514302797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074b40 msgr2=0x7f4acc10e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3514302797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074b40 0x7f4acc10e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009860 tx=0x7f4ab802a3d0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3514302797 shutdown_connections 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3514302797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc10e830 0x7f4acc110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3514302797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074b40 0x7f4acc10e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3514302797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc074220 0x7f4acc074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3514302797 >> 172.21.15.19:0/3514302797 conn(0x7f4acc0793a0 msgr2=0x7f4acc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3514302797 shutdown_connections 2024-04-25T15:04:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.707+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3514302797 wait complete. 2024-04-25T15:04:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 Processor -- start 2024-04-25T15:04:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 -- start start 2024-04-25T15:04:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc074b40 0x7f4acc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc114420 con 0x7f4acc10e830 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1142a0 con 0x7f4acc074b40 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.709+0000 7f4ad1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1145a0 con 0x7f4acc074220 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4aca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc074b40 0x7f4acc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44960/0 (socket says 172.21.15.19:44960) 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55396/0 (socket says 172.21.15.19:55396) 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 -- 172.21.15.19:0/3427471470 learned_addr learned my addr 172.21.15.19:0/3427471470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 -- 172.21.15.19:0/3427471470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc074b40 msgr2=0x7f4acc1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc074b40 0x7f4acc1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 -- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 msgr2=0x7f4acc19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acaa00640 1 -- 172.21.15.19:0/3427471470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19afd0 con 0x7f4acc074220 2024-04-25T15:04:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.710+0000 7f4acb400640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4acaa00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00e9c0 tx=0x7f4abc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00ccc0 con 0x7f4acc074220 2024-04-25T15:04:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc0054d0 con 0x7f4acc074220 2024-04-25T15:04:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19b2c0 con 0x7f4acc074220 2024-04-25T15:04:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc010430 con 0x7f4acc074220 2024-04-25T15:04:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.711+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a74e0 con 0x7f4acc074220 2024-04-25T15:04:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.712+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc075960 con 0x7f4acc074220 2024-04-25T15:04:45.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.714+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4abc004f80 con 0x7f4acc074220 2024-04-25T15:04:45.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.716+0000 7f4ac3400640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 0x7f4ab00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:45.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.716+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4abc0d0de0 con 0x7f4acc074220 2024-04-25T15:04:45.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.716+0000 7f4aca000640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 0x7f4ab00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:45.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.717+0000 7f4aca000640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 0x7f4ab00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4acc19bb50 tx=0x7f4ab8005c90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.720+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4abc09d2e0 con 0x7f4acc074220 2024-04-25T15:04:45.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:45 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/413709765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:45.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:45 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2972838267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:45 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/413709765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:45 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2972838267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:45.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:45 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/413709765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:45.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:45 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2972838267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:45.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.921+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4acc1099b0 con 0x7f4acc074220 2024-04-25T15:04:45.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.922+0000 7f4ac3400640 1 -- 172.21.15.19:0/3427471470 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f4abc09cbe0 con 0x7f4acc074220 2024-04-25T15:04:45.923 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 msgr2=0x7f4ab00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 0x7f4ab00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4acc19bb50 tx=0x7f4ab8005c90 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 msgr2=0x7f4acc1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00e9c0 tx=0x7f4abc00ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 shutdown_connections 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc10e830 0x7f4acc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4acc074b40 0x7f4acc1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f4ab00a3db0 0x7f4ab00a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 --2- 172.21.15.19:0/3427471470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4acc074220 0x7f4acc1a0690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 >> 172.21.15.19:0/3427471470 conn(0x7f4acc0793a0 msgr2=0x7f4acc0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 shutdown_connections 2024-04-25T15:04:45.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:45.924+0000 7f4ad1a00640 1 -- 172.21.15.19:0/3427471470 wait complete. 2024-04-25T15:04:45.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:04:46.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 -- 172.21.15.19:0/1124119482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 msgr2=0x7fcb04111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1124119482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb04111040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009860 tx=0x7fcafc02a340 comp rx=0 tx=0).stop 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 -- 172.21.15.19:0/1124119482 shutdown_connections 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1124119482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb04111040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1124119482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04104d50 0x7fcb04108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1124119482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb041043a0 0x7fcb04104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 -- 172.21.15.19:0/1124119482 >> 172.21.15.19:0/1124119482 conn(0x7fcb040ffc00 msgr2=0x7fcb04102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 -- 172.21.15.19:0/1124119482 shutdown_connections 2024-04-25T15:04:46.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.150+0000 7fcb09c00640 1 -- 172.21.15.19:0/1124119482 wait complete. 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.151+0000 7fcb09c00640 1 Processor -- start 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.151+0000 7fcb09c00640 1 -- start start 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 0x7fcb04076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb04076ee0 0x7fcb041aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb041143c0 con 0x7fcb04076ee0 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04114240 con 0x7fcb041043a0 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb09c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04114540 con 0x7fcb04109470 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb03400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb04076ee0 0x7fcb041aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:44982/0 (socket says 172.21.15.19:44982) 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 -- 172.21.15.19:0/1345867813 learned_addr learned my addr 172.21.15.19:0/1345867813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcb02a00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 0x7fcb04076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 -- 172.21.15.19:0/1345867813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 msgr2=0x7fcb04076460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 0x7fcb04076460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 -- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb04076ee0 msgr2=0x7fcb041aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.152+0000 7fcafa000640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb04076ee0 0x7fcb041aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcafa000640 1 -- 172.21.15.19:0/1345867813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb041ab430 con 0x7fcb04109470 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcb02a00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 0x7fcb04076460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcafa000640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00e9c0 tx=0x7fcaec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00cd80 con 0x7fcb04109470 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec0054d0 con 0x7fcb04109470 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb041ab720 con 0x7fcb04109470 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec010430 con 0x7fcb04109470 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.153+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041abc40 con 0x7fcb04109470 2024-04-25T15:04:46.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.154+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb04105870 con 0x7fcb04109470 2024-04-25T15:04:46.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.157+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fcaec004fd0 con 0x7fcb04109470 2024-04-25T15:04:46.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.158+0000 7fcafbe00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 0x7fcae80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.158+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcaec0d0e20 con 0x7fcb04109470 2024-04-25T15:04:46.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.158+0000 7fcb02a00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 0x7fcae80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.159+0000 7fcb02a00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 0x7fcae80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf40095c0 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:46.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.161+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcaec09d320 con 0x7fcb04109470 2024-04-25T15:04:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.412+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb04068d70 con 0x7fcb04109470 2024-04-25T15:04:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.413+0000 7fcafbe00640 1 -- 172.21.15.19:0/1345867813 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7fcb04068d70 con 0x7fcb04109470 2024-04-25T15:04:46.415 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:46.415 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:04:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:04:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 msgr2=0x7fcae80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 0x7fcae80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf40095c0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 msgr2=0x7fcb040769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00e9c0 tx=0x7fcaec00ee80 comp rx=0 tx=0).stop 2024-04-25T15:04:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 shutdown_connections 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb04076ee0 0x7fcb041aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fcae80a3db0 0x7fcae80a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb04109470 0x7fcb040769a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 --2- 172.21.15.19:0/1345867813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb041043a0 0x7fcb04076460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 >> 172.21.15.19:0/1345867813 conn(0x7fcb040ffc00 msgr2=0x7fcb04100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 shutdown_connections 2024-04-25T15:04:46.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.417+0000 7fcb09c00640 1 -- 172.21.15.19:0/1345867813 wait complete. 2024-04-25T15:04:46.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 -- 172.21.15.19:0/2335944033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41047e0 msgr2=0x7f13a4104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 --2- 172.21.15.19:0/2335944033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41047e0 0x7f13a4104c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f139c009ca0 tx=0x7f139c02a8c0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 -- 172.21.15.19:0/2335944033 shutdown_connections 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 --2- 172.21.15.19:0/2335944033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41047e0 0x7f13a4104c90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 --2- 172.21.15.19:0/2335944033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a4107100 0x7f13a4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 --2- 172.21.15.19:0/2335944033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4106c90 0x7f13a40690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 -- 172.21.15.19:0/2335944033 >> 172.21.15.19:0/2335944033 conn(0x7f13a4076790 msgr2=0x7f13a4076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 -- 172.21.15.19:0/2335944033 shutdown_connections 2024-04-25T15:04:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.641+0000 7f13aae00640 1 -- 172.21.15.19:0/2335944033 wait complete. 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.642+0000 7f13aae00640 1 Processor -- start 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.642+0000 7f13aae00640 1 -- start start 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 0x7f13a41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 0x7f13a419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4118550 con 0x7f13a4106c90 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41183d0 con 0x7f13a4107100 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41186d0 con 0x7f13a41047e0 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13a9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13a9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55434/0 (socket says 172.21.15.19:55434) 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13a9000640 1 -- 172.21.15.19:0/848885421 learned_addr learned my addr 172.21.15.19:0/848885421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13a9a00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 0x7f13a41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.643+0000 7f13aa400640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 0x7f13a419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 -- 172.21.15.19:0/848885421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 msgr2=0x7f13a41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 0x7f13a41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 -- 172.21.15.19:0/848885421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 msgr2=0x7f13a419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 0x7f13a419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 -- 172.21.15.19:0/848885421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a419f050 con 0x7f13a4106c90 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13aa400640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 0x7f13a419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9a00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 0x7f13a41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a9000640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f139000ea60 tx=0x7f139000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139000cde0 con 0x7f13a4106c90 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13900035f0 con 0x7f13a4106c90 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390010690 con 0x7f13a4106c90 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a419f340 con 0x7f13a4106c90 2024-04-25T15:04:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.644+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41ab590 con 0x7f13a4106c90 2024-04-25T15:04:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.646+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4068c90 con 0x7f13a4106c90 2024-04-25T15:04:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.649+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f13900037a0 con 0x7f13a4106c90 2024-04-25T15:04:46.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.650+0000 7f13a2000640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 0x7f138c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:46.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.650+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f13900d0bb0 con 0x7f13a4106c90 2024-04-25T15:04:46.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.650+0000 7f13a9a00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 0x7f138c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:46.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.651+0000 7f13a9a00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 0x7f138c0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f13a4101a80 tx=0x7f1398005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.654+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f139009d110 con 0x7f13a4106c90 2024-04-25T15:04:46.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:46 smithi083 ceph-mon[32919]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3425192628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3427471470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:46 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1345867813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:46 smithi102 ceph-mon[33005]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3425192628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3427471470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:46 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1345867813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:46.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.865+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13a41ab930 con 0x7f13a4106c90 2024-04-25T15:04:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.868+0000 7f13a2000640 1 -- 172.21.15.19:0/848885421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22954 (secure 0 0 0) 0x7f139009ca10 con 0x7f13a4106c90 2024-04-25T15:04:46.869 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:46.869 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:04:29.064393+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:46.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 msgr2=0x7f138c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 0x7f138c0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f13a4101a80 tx=0x7f1398005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:46.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 msgr2=0x7f13a41a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:46 smithi019 ceph-mon[28090]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-04-25T15:04:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3425192628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3427471470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:46.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:46 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1345867813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:46.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f139000ea60 tx=0x7f139000ef20 comp rx=0 tx=0).stop 2024-04-25T15:04:46.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 shutdown_connections 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4107100 0x7f13a419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.870+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4106c90 0x7f13a41a4c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.871+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f138c0a3e30 0x7f138c0a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.871+0000 7f13aae00640 1 --2- 172.21.15.19:0/848885421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13a41047e0 0x7f13a41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.871+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 >> 172.21.15.19:0/848885421 conn(0x7f13a4076790 msgr2=0x7f13a4101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:46.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.871+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 shutdown_connections 2024-04-25T15:04:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:46.871+0000 7f13aae00640 1 -- 172.21.15.19:0/848885421 wait complete. 2024-04-25T15:04:46.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:04:46.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:04:47.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 -- 172.21.15.19:0/3887376252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040102070 msgr2=0x7f5040102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 --2- 172.21.15.19:0/3887376252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040102070 0x7f5040102450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5030009900 tx=0x7f503002a410 comp rx=0 tx=0).stop 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 -- 172.21.15.19:0/3887376252 shutdown_connections 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 --2- 172.21.15.19:0/3887376252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f504010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 --2- 172.21.15.19:0/3887376252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102a20 0x7f5040106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 --2- 172.21.15.19:0/3887376252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040102070 0x7f5040102450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 -- 172.21.15.19:0/3887376252 >> 172.21.15.19:0/3887376252 conn(0x7f50400fd8f0 msgr2=0x7f50400ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 -- 172.21.15.19:0/3887376252 shutdown_connections 2024-04-25T15:04:47.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.115+0000 7f503f200640 1 -- 172.21.15.19:0/3887376252 wait complete. 2024-04-25T15:04:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 Processor -- start 2024-04-25T15:04:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 -- start start 2024-04-25T15:04:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 0x7f5040074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f5040074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50401120d0 con 0x7f5040107140 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111f50 con 0x7f5040102070 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.116+0000 7f503f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5040112250 con 0x7f5040074cd0 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f5040074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:45034/0 (socket says 172.21.15.19:45034) 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 -- 172.21.15.19:0/3785065073 learned_addr learned my addr 172.21.15.19:0/3785065073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f5040074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55466/0 (socket says 172.21.15.19:55466) 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503de00640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 0x7f5040074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 -- 172.21.15.19:0/3785065073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 msgr2=0x7f5040074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 0x7f5040074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 -- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 msgr2=0x7f5040074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f5040074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503e800640 1 -- 172.21.15.19:0/3785065073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401a6ee0 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.117+0000 7f503de00640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 0x7f5040074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f503e800640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f503800eb10 tx=0x7f503800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038004b20 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5038004cc0 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401a71d0 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038010670 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.118+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a76d0 con 0x7f5040074cd0 2024-04-25T15:04:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.119+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50400690e0 con 0x7f5040074cd0 2024-04-25T15:04:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.121+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f5038005020 con 0x7f5040074cd0 2024-04-25T15:04:47.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.123+0000 7f5036a00640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 0x7f50240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.123+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f50380d0ff0 con 0x7f5040074cd0 2024-04-25T15:04:47.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.123+0000 7f503de00640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 0x7f50240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.124+0000 7f503de00640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 0x7f50240a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c00 tx=0x7f5030005a20 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:47.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.125+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f503809d550 con 0x7f5040074cd0 2024-04-25T15:04:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.331+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f504010a370 con 0x7f5040074cd0 2024-04-25T15:04:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.332+0000 7f5036a00640 1 -- 172.21.15.19:0/3785065073 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f503809ce50 con 0x7f5040074cd0 2024-04-25T15:04:47.334 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 msgr2=0x7f50240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 0x7f50240a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c00 tx=0x7f5030005a20 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 msgr2=0x7f50401a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f503800eb10 tx=0x7f503800c2d0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 shutdown_connections 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f50240a3db0 0x7f50240a6270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5040074cd0 0x7f50401a6840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040107140 0x7f5040074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 --2- 172.21.15.19:0/3785065073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5040102070 0x7f5040074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 >> 172.21.15.19:0/3785065073 conn(0x7f50400fd8f0 msgr2=0x7f50400fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.335+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 shutdown_connections 2024-04-25T15:04:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.336+0000 7f503f200640 1 -- 172.21.15.19:0/3785065073 wait complete. 2024-04-25T15:04:47.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 -- 172.21.15.19:0/725789272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a58104d50 msgr2=0x7f7a58108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 --2- 172.21.15.19:0/725789272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a58104d50 0x7f7a58108d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009860 tx=0x7f7a4402a340 comp rx=0 tx=0).stop 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 -- 172.21.15.19:0/725789272 shutdown_connections 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 --2- 172.21.15.19:0/725789272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a58111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 --2- 172.21.15.19:0/725789272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a58104d50 0x7f7a58108d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 --2- 172.21.15.19:0/725789272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a58104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 -- 172.21.15.19:0/725789272 >> 172.21.15.19:0/725789272 conn(0x7f7a580ffbc0 msgr2=0x7f7a58101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 -- 172.21.15.19:0/725789272 shutdown_connections 2024-04-25T15:04:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.578+0000 7f7a5d800640 1 -- 172.21.15.19:0/725789272 wait complete. 2024-04-25T15:04:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 Processor -- start 2024-04-25T15:04:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 -- start start 2024-04-25T15:04:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a581a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58114390 con 0x7f7a581043a0 2024-04-25T15:04:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58114210 con 0x7f7a58109470 2024-04-25T15:04:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.580+0000 7f7a5d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58114510 con 0x7f7a5819a7a0 2024-04-25T15:04:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a581a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:45054/0 (socket says 172.21.15.19:45054) 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a57400640 1 -- 172.21.15.19:0/2966058844 learned_addr learned my addr 172.21.15.19:0/2966058844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:44990/0 (socket says 172.21.15.19:44990) 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 -- 172.21.15.19:0/2966058844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 msgr2=0x7f7a5819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 -- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 msgr2=0x7f7a581a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a581a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56000640 1 -- 172.21.15.19:0/2966058844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5819b330 con 0x7f7a58109470 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a57400640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.581+0000 7f7a56a00640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a581a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a56000640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a44035040 tx=0x7f7a44032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44040020 con 0x7f7a58109470 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a44030450 con 0x7f7a58109470 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44037040 con 0x7f7a58109470 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5819b5c0 con 0x7f7a58109470 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.582+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a581a7720 con 0x7f7a58109470 2024-04-25T15:04:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.583+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a58105870 con 0x7f7a58109470 2024-04-25T15:04:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.585+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7a44043020 con 0x7f7a58109470 2024-04-25T15:04:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.586+0000 7f7a4f400640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 0x7f7a3c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.587+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f7a440f0450 con 0x7f7a58109470 2024-04-25T15:04:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.587+0000 7f7a56a00640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 0x7f7a3c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.587+0000 7f7a56a00640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 0x7f7a3c0a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a48005fd0 tx=0x7f7a48010040 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:47.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.591+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a4404a020 con 0x7f7a58109470 2024-04-25T15:04:47.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:47 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/848885421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:47.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:47 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3785065073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:47.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:47 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/848885421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:47.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:47 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3785065073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.799+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a58104d50 con 0x7f7a58109470 2024-04-25T15:04:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.800+0000 7f7a4f400640 1 -- 172.21.15.19:0/2966058844 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f7a440bc410 con 0x7f7a58109470 2024-04-25T15:04:47.801 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 msgr2=0x7f7a3c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 0x7f7a3c0a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a48005fd0 tx=0x7f7a48010040 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 msgr2=0x7f7a581a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a44035040 tx=0x7f7a44032dc0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 shutdown_connections 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7a3c0a3e30 0x7f7a3c0a62f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5819a7a0 0x7f7a5819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a58109470 0x7f7a581a0b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 --2- 172.21.15.19:0/2966058844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a581043a0 0x7f7a581a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 >> 172.21.15.19:0/2966058844 conn(0x7f7a580ffbc0 msgr2=0x7f7a58100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 shutdown_connections 2024-04-25T15:04:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:47.803+0000 7f7a5d800640 1 -- 172.21.15.19:0/2966058844 wait complete. 2024-04-25T15:04:47.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:47 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/848885421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:47.809 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:47 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3785065073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:04:47.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:04:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 -- 172.21.15.19:0/1706689770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac074c10 msgr2=0x7f7dac1128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 --2- 172.21.15.19:0/1706689770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac074c10 0x7f7dac1128c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009900 tx=0x7f7d9402a480 comp rx=0 tx=0).stop 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 -- 172.21.15.19:0/1706689770 shutdown_connections 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 --2- 172.21.15.19:0/1706689770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac112e00 0x7f7dac1151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 --2- 172.21.15.19:0/1706689770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac074c10 0x7f7dac1128c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 --2- 172.21.15.19:0/1706689770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 -- 172.21.15.19:0/1706689770 >> 172.21.15.19:0/1706689770 conn(0x7f7dac0fd920 msgr2=0x7f7dac0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 -- 172.21.15.19:0/1706689770 shutdown_connections 2024-04-25T15:04:48.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.046+0000 7f7db2000640 1 -- 172.21.15.19:0/1706689770 wait complete. 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.047+0000 7f7db2000640 1 Processor -- start 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.047+0000 7f7db2000640 1 -- start start 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 0x7f7dac1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac1186d0 con 0x7f7dac074c10 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac118550 con 0x7f7dac0742f0 2024-04-25T15:04:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac118850 con 0x7f7dac112e00 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 0x7f7dac1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:45058/0 (socket says 172.21.15.19:45058) 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.048+0000 7f7db1600640 1 -- 172.21.15.19:0/2606481059 learned_addr learned my addr 172.21.15.19:0/2606481059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 -- 172.21.15.19:0/2606481059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 msgr2=0x7f7dac1a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac1a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 -- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 msgr2=0x7f7dac1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 0x7f7dac1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 -- 172.21.15.19:0/2606481059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac19f230 con 0x7f7dac112e00 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db0c00640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7da3e00640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 0x7f7dac1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db1600640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7da400dac0 tx=0x7f7da400df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4017070 con 0x7f7dac112e00 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac19f520 con 0x7f7dac112e00 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.049+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da400be30 con 0x7f7dac112e00 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.050+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4010460 con 0x7f7dac112e00 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.050+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1ab770 con 0x7f7dac112e00 2024-04-25T15:04:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.051+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac068c90 con 0x7f7dac112e00 2024-04-25T15:04:48.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.053+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7da40040b0 con 0x7f7dac112e00 2024-04-25T15:04:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.055+0000 7f7daa000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 0x7f7d900a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.055+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f7da40d03c0 con 0x7f7dac112e00 2024-04-25T15:04:48.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.055+0000 7f7db0c00640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 0x7f7d900a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.056+0000 7f7db0c00640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 0x7f7d900a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dac115460 tx=0x7f7d9c005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:48.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.057+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da409c2f0 con 0x7f7dac112e00 2024-04-25T15:04:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.262+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7dac1052f0 con 0x7f7dac112e00 2024-04-25T15:04:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.263+0000 7f7daa000640 1 -- 172.21.15.19:0/2606481059 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f7da40a05f0 con 0x7f7dac112e00 2024-04-25T15:04:48.265 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:04:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 msgr2=0x7f7d900a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 0x7f7d900a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dac115460 tx=0x7f7d9c005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 msgr2=0x7f7dac19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7da400dac0 tx=0x7f7da400df80 comp rx=0 tx=0).stop 2024-04-25T15:04:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 shutdown_connections 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f7d900a3e30 0x7f7d900a62f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dac112e00 0x7f7dac19eab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dac074c10 0x7f7dac1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 --2- 172.21.15.19:0/2606481059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dac0742f0 0x7f7dac1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 >> 172.21.15.19:0/2606481059 conn(0x7f7dac0fd920 msgr2=0x7f7dac0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 shutdown_connections 2024-04-25T15:04:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.266+0000 7f7db2000640 1 -- 172.21.15.19:0/2606481059 wait complete. 2024-04-25T15:04:48.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:04:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.508+0000 7f46f2e00640 1 -- 172.21.15.19:0/1010305230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec10e640 msgr2=0x7f46ec10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.508+0000 7f46f2e00640 1 --2- 172.21.15.19:0/1010305230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec10e640 0x7f46ec10ea20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009900 tx=0x7f46dc02a410 comp rx=0 tx=0).stop 2024-04-25T15:04:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 -- 172.21.15.19:0/1010305230 shutdown_connections 2024-04-25T15:04:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 --2- 172.21.15.19:0/1010305230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec108f40 0x7f46ec115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 --2- 172.21.15.19:0/1010305230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108580 0x7f46ec108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 --2- 172.21.15.19:0/1010305230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec10e640 0x7f46ec10ea20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 -- 172.21.15.19:0/1010305230 >> 172.21.15.19:0/1010305230 conn(0x7f46ec075750 msgr2=0x7f46ec106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 -- 172.21.15.19:0/1010305230 shutdown_connections 2024-04-25T15:04:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.509+0000 7f46f2e00640 1 -- 172.21.15.19:0/1010305230 wait complete. 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 Processor -- start 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 -- start start 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108f40 0x7f46ec19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 0x7f46ec1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1187d0 con 0x7f46ec108580 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec118650 con 0x7f46ec108f40 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46f2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec118950 con 0x7f46ec10e640 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.510+0000 7f46ebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55522/0 (socket says 172.21.15.19:55522) 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 -- 172.21.15.19:0/2257972840 learned_addr learned my addr 172.21.15.19:0/2257972840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46e3e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108f40 0x7f46ec19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46f0e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 0x7f46ec1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 -- 172.21.15.19:0/2257972840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 msgr2=0x7f46ec1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 0x7f46ec1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 -- 172.21.15.19:0/2257972840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108f40 msgr2=0x7f46ec19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108f40 0x7f46ec19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 -- 172.21.15.19:0/2257972840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1a3d40 con 0x7f46ec108580 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46f0e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 0x7f46ec1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.511+0000 7f46ebe00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006010 tx=0x7f46dc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.512+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc03f030 con 0x7f46ec108580 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.512+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46dc0044c0 con 0x7f46ec108580 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.512+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc035070 con 0x7f46ec108580 2024-04-25T15:04:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.512+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a3fd0 con 0x7f46ec108580 2024-04-25T15:04:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.512+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1ab8c0 con 0x7f46ec108580 2024-04-25T15:04:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.513+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec109ca0 con 0x7f46ec108580 2024-04-25T15:04:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.515+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f46dc042030 con 0x7f46ec108580 2024-04-25T15:04:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.517+0000 7f46e9600640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 0x7f46d00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.517+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f46dc0f0c00 con 0x7f46ec108580 2024-04-25T15:04:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.517+0000 7f46e3e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 0x7f46d00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.518+0000 7f46e3e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 0x7f46d00a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f46d40095f0 tx=0x7f46d4005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.521+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46dc0bcab0 con 0x7f46ec108580 2024-04-25T15:04:48.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:48 smithi019 ceph-mon[28090]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:04:48.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:48 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2966058844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:48.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:48 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2606481059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:48.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.775+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ec110a30 con 0x7f46ec108580 2024-04-25T15:04:48.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.776+0000 7f46e9600640 1 -- 172.21.15.19:0/2257972840 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7f46dc0c0db0 con 0x7f46ec108580 2024-04-25T15:04:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:48.778 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:04:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:04:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.779+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 msgr2=0x7f46d00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.779+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 0x7f46d00a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f46d40095f0 tx=0x7f46d4005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.779+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 msgr2=0x7f46ec19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:48.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.779+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46dc006010 tx=0x7f46dc0040c0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 shutdown_connections 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f46d00a3e30 0x7f46d00a62f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46ec10e640 0x7f46ec1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46ec108f40 0x7f46ec19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 --2- 172.21.15.19:0/2257972840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46ec108580 0x7f46ec19ed50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:48.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 >> 172.21.15.19:0/2257972840 conn(0x7f46ec075750 msgr2=0x7f46ec076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 shutdown_connections 2024-04-25T15:04:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:48.780+0000 7f46f2e00640 1 -- 172.21.15.19:0/2257972840 wait complete. 2024-04-25T15:04:48.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:48 smithi083 ceph-mon[32919]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:04:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:48 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2966058844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:48 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2606481059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:48.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:04:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:48 smithi102 ceph-mon[33005]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:04:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:48 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2966058844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:04:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:48 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2606481059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 -- 172.21.15.19:0/2651044881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 msgr2=0x7f63c01045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 --2- 172.21.15.19:0/2651044881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01045a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009900 tx=0x7f63b002a410 comp rx=0 tx=0).stop 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 -- 172.21.15.19:0/2651044881 shutdown_connections 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 --2- 172.21.15.19:0/2651044881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c0109290 0x7f63c0110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 --2- 172.21.15.19:0/2651044881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0104b70 0x7f63c0108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 --2- 172.21.15.19:0/2651044881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01045a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.018+0000 7f63bf200640 1 -- 172.21.15.19:0/2651044881 >> 172.21.15.19:0/2651044881 conn(0x7f63c00ffa20 msgr2=0x7f63c0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.019+0000 7f63bf200640 1 -- 172.21.15.19:0/2651044881 shutdown_connections 2024-04-25T15:04:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.019+0000 7f63bf200640 1 -- 172.21.15.19:0/2651044881 wait complete. 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.020+0000 7f63bf200640 1 Processor -- start 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.020+0000 7f63bf200640 1 -- start start 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.020+0000 7f63bf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.020+0000 7f63bf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0109290 0x7f63c01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.020+0000 7f63bf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c019a600 0x7f63c019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0114220 con 0x7f63c01041c0 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01140a0 con 0x7f63c019a600 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01143a0 con 0x7f63c0109290 2024-04-25T15:04:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:49.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0109290 0x7f63c01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:49.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55540/0 (socket says 172.21.15.19:55540) 2024-04-25T15:04:49.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 -- 172.21.15.19:0/418077615 learned_addr learned my addr 172.21.15.19:0/418077615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:04:49.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63be800640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c019a600 0x7f63c019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 -- 172.21.15.19:0/418077615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0109290 msgr2=0x7f63c01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0109290 0x7f63c01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 -- 172.21.15.19:0/418077615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c019a600 msgr2=0x7f63c019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c019a600 0x7f63c019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.021+0000 7f63bde00640 1 -- 172.21.15.19:0/418077615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c019b120 con 0x7f63c01041c0 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63bde00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005ab0 tx=0x7f63b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b003f070 con 0x7f63c01041c0 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b0031b60 con 0x7f63c01041c0 2024-04-25T15:04:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c019b3b0 con 0x7f63c01041c0 2024-04-25T15:04:49.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0035030 con 0x7f63c01041c0 2024-04-25T15:04:49.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.022+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01a75b0 con 0x7f63c01041c0 2024-04-25T15:04:49.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.023+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c00690e0 con 0x7f63c01041c0 2024-04-25T15:04:49.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.026+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f63b0042070 con 0x7f63c01041c0 2024-04-25T15:04:49.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.027+0000 7f63b6a00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 0x7f63a40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:04:49.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.027+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f63b00f1600 con 0x7f63c01041c0 2024-04-25T15:04:49.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.027+0000 7f63b7e00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 0x7f63a40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:04:49.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.028+0000 7f63b7e00640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 0x7f63a40a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63c0103170 tx=0x7f63a8005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:04:49.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.030+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63b00bd4b0 con 0x7f63c01041c0 2024-04-25T15:04:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.237+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63c0104b70 con 0x7f63c01041c0 2024-04-25T15:04:49.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.242+0000 7f63b6a00640 1 -- 172.21.15.19:0/418077615 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22954 (secure 0 0 0) 0x7f63b00c17b0 con 0x7f63c01041c0 2024-04-25T15:04:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:04:49.244 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:04:29.064393+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.244+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 msgr2=0x7f63a40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.244+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 0x7f63a40a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63c0103170 tx=0x7f63a8005f90 comp rx=0 tx=0).stop 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.244+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 msgr2=0x7f63c01a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.244+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005ab0 tx=0x7f63b0003c10 comp rx=0 tx=0).stop 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 shutdown_connections 2024-04-25T15:04:49.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63c019a600 0x7f63c019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f63a40a3e30 0x7f63a40a62f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c0109290 0x7f63c01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 --2- 172.21.15.19:0/418077615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c01041c0 0x7f63c01a0400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 >> 172.21.15.19:0/418077615 conn(0x7f63c00ffa20 msgr2=0x7f63c0100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 shutdown_connections 2024-04-25T15:04:49.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:04:49.245+0000 7f63bf200640 1 -- 172.21.15.19:0/418077615 wait complete. 2024-04-25T15:04:49.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir 2024-04-25T15:04:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-25T15:04:49.469 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T15:04:49.469 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T15:04:49.711 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=eb637484bd754558891242e4e7789d25 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T15:04:49.715 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T15:04:49.726 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T15:04:49.735 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:49.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:49 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2257972840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:49 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/418077615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.763 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T15:04:49.773 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T15:04:49.773 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:04:49.773 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:04:49.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T15:04:49.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:49 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2257972840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:49 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/418077615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:49 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2257972840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:49 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/418077615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:04:49.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T15:04:50.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:50 smithi083 ceph-mon[32919]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:50 smithi102 ceph-mon[33005]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:50 smithi019 ceph-mon[28090]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:52.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:52 smithi083 ceph-mon[32919]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:04:52.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:52 smithi102 ceph-mon[33005]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:04:52.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:52 smithi019 ceph-mon[28090]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:04:54.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:54 smithi083 ceph-mon[32919]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:54 smithi102 ceph-mon[33005]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:54.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:54 smithi019 ceph-mon[28090]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:56 smithi083 ceph-mon[32919]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:56 smithi102 ceph-mon[33005]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:56.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:56 smithi019 ceph-mon[28090]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:04:58.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:04:58 smithi083 ceph-mon[32919]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:04:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:04:58 smithi102 ceph-mon[33005]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:04:58.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:04:58 smithi019 ceph-mon[28090]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:05:00.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-mon[28090]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:05:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:00 smithi083 ceph-mon[32919]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:05:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:00 smithi102 ceph-mon[33005]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:05:01.122 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:05:00.835+0000 7f497c000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:05:01.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:05:00.835+0000 7f497c000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:05:01.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:05:00.835+0000 7f497c000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:05:01.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:05:00.835+0000 7f497c000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:05:01.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:05:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:05:00.835+0000 7f497c000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:05:02.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:02 smithi083 ceph-mon[32919]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:05:02.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:02 smithi083 ceph-mon[32919]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-25T15:05:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:02 smithi102 ceph-mon[33005]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:05:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:02 smithi102 ceph-mon[33005]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-25T15:05:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:02 smithi019 ceph-mon[28090]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:05:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:02 smithi019 ceph-mon[28090]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-25T15:05:04.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:04 smithi083 ceph-mon[32919]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:04.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:04 smithi102 ceph-mon[33005]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:04.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:04 smithi019 ceph-mon[28090]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:06 smithi083 ceph-mon[32919]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:06 smithi102 ceph-mon[33005]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:06.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:06 smithi019 ceph-mon[28090]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:08.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:08 smithi083 ceph-mon[32919]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:08 smithi102 ceph-mon[33005]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:08.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:08 smithi019 ceph-mon[28090]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:10.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:10 smithi083 ceph-mon[32919]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:10 smithi102 ceph-mon[33005]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:10.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:10 smithi019 ceph-mon[28090]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:12.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:12 smithi083 ceph-mon[32919]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:12.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:12 smithi102 ceph-mon[33005]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:12 smithi019 ceph-mon[28090]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:14 smithi102 ceph-mon[33005]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:14.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:14 smithi019 ceph-mon[28090]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:15.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:14 smithi083 ceph-mon[32919]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:16 smithi102 ceph-mon[33005]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:16 smithi019 ceph-mon[28090]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:16 smithi083 ceph-mon[32919]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:18 smithi102 ceph-mon[33005]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:18 smithi019 ceph-mon[28090]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:18 smithi083 ceph-mon[32919]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:20.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:20 smithi083 ceph-mon[32919]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:20 smithi019 ceph-mon[28090]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:21.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:20 smithi102 ceph-mon[33005]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:22 smithi019 ceph-mon[28090]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:23.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:22 smithi083 ceph-mon[32919]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:22 smithi102 ceph-mon[33005]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:24 smithi019 ceph-mon[28090]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:25.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:24 smithi083 ceph-mon[32919]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:25.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:24 smithi102 ceph-mon[33005]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:26.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:26 smithi019 ceph-mon[28090]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:27.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:26 smithi083 ceph-mon[32919]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:27.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:26 smithi102 ceph-mon[33005]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:28.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:28 smithi019 ceph-mon[28090]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:29.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:28 smithi083 ceph-mon[32919]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:29.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:28 smithi102 ceph-mon[33005]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:30.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:30 smithi019 ceph-mon[28090]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:31.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:30 smithi083 ceph-mon[32919]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:31.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:30 smithi102 ceph-mon[33005]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:32 smithi019 ceph-mon[28090]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:33.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:32 smithi083 ceph-mon[32919]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:32 smithi102 ceph-mon[33005]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:34.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:34 smithi019 ceph-mon[28090]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:35.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:34 smithi083 ceph-mon[32919]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:35.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:34 smithi102 ceph-mon[33005]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:36 smithi019 ceph-mon[28090]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:37.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:36 smithi083 ceph-mon[32919]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:37.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:36 smithi102 ceph-mon[33005]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:38 smithi019 ceph-mon[28090]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:38 smithi083 ceph-mon[32919]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:39.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:38 smithi102 ceph-mon[33005]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:40 smithi019 ceph-mon[28090]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:40 smithi083 ceph-mon[32919]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:40 smithi102 ceph-mon[33005]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:41 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:42.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:41 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:42.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:41 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:43.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:42 smithi083 ceph-mon[32919]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:43.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:42 smithi102 ceph-mon[33005]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:43.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:42 smithi019 ceph-mon[28090]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:43 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:43 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:43 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:05:44.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:43 smithi083 ceph-mon[32919]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:44.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:43 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:44.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:43 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:44.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:43 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:05:44.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:43 smithi102 ceph-mon[33005]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:43 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:43 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:43 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:05:44.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:43 smithi019 ceph-mon[28090]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:46.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:46 smithi083 ceph-mon[32919]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:46 smithi102 ceph-mon[33005]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:46.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:46 smithi019 ceph-mon[28090]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:48.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:48 smithi083 ceph-mon[32919]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:48.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:48 smithi102 ceph-mon[33005]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:48.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:48 smithi019 ceph-mon[28090]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:50.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:50 smithi083 ceph-mon[32919]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:50.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:50 smithi102 ceph-mon[33005]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:50.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:50 smithi019 ceph-mon[28090]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:52.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:52 smithi083 ceph-mon[32919]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:52 smithi102 ceph-mon[33005]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:52.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:52 smithi019 ceph-mon[28090]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:54.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:54 smithi083 ceph-mon[32919]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:54.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:54 smithi102 ceph-mon[33005]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:54.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:54 smithi019 ceph-mon[28090]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:56.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:56 smithi083 ceph-mon[32919]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:56.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:56 smithi102 ceph-mon[33005]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:56.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:56 smithi019 ceph-mon[28090]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:58.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:05:58 smithi083 ceph-mon[32919]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:58.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:05:58 smithi102 ceph-mon[33005]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:58.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:05:58 smithi019 ceph-mon[28090]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:00.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:00 smithi019 ceph-mon[28090]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:00.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:00 smithi083 ceph-mon[32919]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:00.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:00 smithi102 ceph-mon[33005]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:02.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:02 smithi083 ceph-mon[32919]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:02.542 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:02 smithi083 ceph-mon[32919]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T15:06:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:02 smithi102 ceph-mon[33005]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:02.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:02 smithi102 ceph-mon[33005]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T15:06:02.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:02 smithi019 ceph-mon[28090]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:02.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:02 smithi019 ceph-mon[28090]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T15:06:04.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:04 smithi083 ceph-mon[32919]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:04.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:04 smithi102 ceph-mon[33005]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:04.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:04 smithi019 ceph-mon[28090]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:06:06.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:06 smithi083 ceph-mon[32919]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:06.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:06 smithi102 ceph-mon[33005]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:06 smithi019 ceph-mon[28090]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:08.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:08 smithi083 ceph-mon[32919]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:08.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:08 smithi102 ceph-mon[33005]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:08.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:08 smithi019 ceph-mon[28090]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:10.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:10 smithi083 ceph-mon[32919]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:10.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:10 smithi102 ceph-mon[33005]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:10.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:10 smithi019 ceph-mon[28090]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:12.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:12 smithi083 ceph-mon[32919]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T15:06:12.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:12 smithi102 ceph-mon[33005]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T15:06:12.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:12 smithi019 ceph-mon[28090]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T15:06:14.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:14 smithi083 ceph-mon[32919]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:14.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:14 smithi102 ceph-mon[33005]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:14 smithi019 ceph-mon[28090]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:16.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:16 smithi083 ceph-mon[32919]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:16.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:16 smithi102 ceph-mon[33005]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:16.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:16 smithi019 ceph-mon[28090]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:18.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:18 smithi083 ceph-mon[32919]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:18.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:18 smithi102 ceph-mon[33005]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:18.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:18 smithi019 ceph-mon[28090]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:20.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:20 smithi102 ceph-mon[33005]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:20 smithi019 ceph-mon[28090]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:20.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:20 smithi083 ceph-mon[32919]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:22 smithi102 ceph-mon[33005]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:22.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:22 smithi019 ceph-mon[28090]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:22.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:22 smithi083 ceph-mon[32919]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:24.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:24 smithi102 ceph-mon[33005]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:24 smithi019 ceph-mon[28090]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:24.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:24 smithi083 ceph-mon[32919]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:26.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:26 smithi019 ceph-mon[28090]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:26.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:26 smithi083 ceph-mon[32919]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:26 smithi102 ceph-mon[33005]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:28.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:28 smithi019 ceph-mon[28090]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:28.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:28 smithi083 ceph-mon[32919]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:28.827 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:28 smithi102 ceph-mon[33005]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:30.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:30 smithi019 ceph-mon[28090]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:30.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:30 smithi083 ceph-mon[32919]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:30.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:30 smithi102 ceph-mon[33005]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:32.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:32 smithi019 ceph-mon[28090]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:32.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:32 smithi083 ceph-mon[32919]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:32 smithi102 ceph-mon[33005]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:34.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:34 smithi019 ceph-mon[28090]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:34.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:34 smithi083 ceph-mon[32919]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:34 smithi102 ceph-mon[33005]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:36.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:36 smithi019 ceph-mon[28090]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:36.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:36 smithi083 ceph-mon[32919]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:36.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:36 smithi102 ceph-mon[33005]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:38.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:38 smithi019 ceph-mon[28090]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:38.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:38 smithi083 ceph-mon[32919]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:38.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:38 smithi102 ceph-mon[33005]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:40.610 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:40 smithi102 ceph-mon[33005]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:40.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:40 smithi019 ceph-mon[28090]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:40.795 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:40 smithi083 ceph-mon[32919]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:42 smithi019 ceph-mon[28090]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:42.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:42 smithi083 ceph-mon[32919]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:42 smithi102 ceph-mon[33005]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:43.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:43 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:43.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:43 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:43.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:43 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:44.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:44 smithi019 ceph-mon[28090]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:44.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:44 smithi083 ceph-mon[32919]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:44 smithi102 ceph-mon[33005]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:45.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:45 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:45.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:45 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:45.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:45 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:06:45.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:45 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:45.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:45 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:45.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:45 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:06:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:45 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:45 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:45.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:45 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:06:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:46 smithi083 ceph-mon[32919]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:46 smithi102 ceph-mon[33005]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:46 smithi019 ceph-mon[28090]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:48 smithi083 ceph-mon[32919]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:48 smithi102 ceph-mon[33005]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:48.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:48 smithi019 ceph-mon[28090]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:50.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:50 smithi083 ceph-mon[32919]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:50 smithi102 ceph-mon[33005]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:50 smithi019 ceph-mon[28090]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:52.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:52 smithi102 ceph-mon[33005]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:52.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:52 smithi019 ceph-mon[28090]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:52 smithi083 ceph-mon[32919]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:54 smithi102 ceph-mon[33005]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:54.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:54 smithi019 ceph-mon[28090]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:55.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:54 smithi083 ceph-mon[32919]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:56 smithi102 ceph-mon[33005]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:56.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:56 smithi019 ceph-mon[28090]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:56 smithi083 ceph-mon[32919]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:58.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:06:58 smithi019 ceph-mon[28090]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:59.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:06:58 smithi083 ceph-mon[32919]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:06:58 smithi102 ceph-mon[33005]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:00.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:00 smithi019 ceph-mon[28090]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:01.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:00 smithi083 ceph-mon[32919]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:00 smithi102 ceph-mon[33005]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:02.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:02 smithi019 ceph-mon[28090]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:03.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:02 smithi083 ceph-mon[32919]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:03.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:02 smithi102 ceph-mon[33005]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:04.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:04 smithi019 ceph-mon[28090]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:04 smithi083 ceph-mon[32919]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:05.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:04 smithi102 ceph-mon[33005]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:06.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:06 smithi019 ceph-mon[28090]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:07.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:06 smithi083 ceph-mon[32919]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:07.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:06 smithi102 ceph-mon[33005]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:08.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:08 smithi019 ceph-mon[28090]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:09.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:08 smithi083 ceph-mon[32919]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:09.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:08 smithi102 ceph-mon[33005]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:10.860 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:10 smithi102 ceph-mon[33005]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:10.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:10 smithi019 ceph-mon[28090]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:11.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:10 smithi083 ceph-mon[32919]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:12 smithi019 ceph-mon[28090]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:12 smithi083 ceph-mon[32919]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:12 smithi102 ceph-mon[33005]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:14.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:14 smithi019 ceph-mon[28090]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:14 smithi083 ceph-mon[32919]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:15.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:14 smithi102 ceph-mon[33005]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:16 smithi019 ceph-mon[28090]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:17.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:16 smithi083 ceph-mon[32919]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:16 smithi102 ceph-mon[33005]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:18 smithi019 ceph-mon[28090]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:19.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:18 smithi083 ceph-mon[32919]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:18 smithi102 ceph-mon[33005]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:20 smithi019 ceph-mon[28090]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:20 smithi083 ceph-mon[32919]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:21.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:20 smithi102 ceph-mon[33005]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:23.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:22 smithi083 ceph-mon[32919]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:22 smithi102 ceph-mon[33005]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:23.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:22 smithi019 ceph-mon[28090]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:25.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:24 smithi083 ceph-mon[32919]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:25.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:24 smithi102 ceph-mon[33005]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:25.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:24 smithi019 ceph-mon[28090]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:27.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:26 smithi083 ceph-mon[32919]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:27.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:26 smithi102 ceph-mon[33005]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:27.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:26 smithi019 ceph-mon[28090]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:28.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:27 smithi019 ceph-mon[28090]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:28.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:27 smithi083 ceph-mon[32919]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:28.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:27 smithi102 ceph-mon[33005]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:30.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:30 smithi083 ceph-mon[32919]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:30.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:30 smithi102 ceph-mon[33005]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:30.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:30 smithi019 ceph-mon[28090]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:32.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:32 smithi083 ceph-mon[32919]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:32.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:32 smithi102 ceph-mon[33005]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:32.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:32 smithi019 ceph-mon[28090]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:34.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:34 smithi083 ceph-mon[32919]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:34.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:34 smithi102 ceph-mon[33005]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:34.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:34 smithi019 ceph-mon[28090]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:36.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:36 smithi083 ceph-mon[32919]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:36.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:36 smithi102 ceph-mon[33005]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:36.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:36 smithi019 ceph-mon[28090]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:38.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:38 smithi083 ceph-mon[32919]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:38.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:38 smithi102 ceph-mon[33005]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:38.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:38 smithi019 ceph-mon[28090]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:40.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:40 smithi083 ceph-mon[32919]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:40.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:40 smithi102 ceph-mon[33005]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:40.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:40 smithi019 ceph-mon[28090]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:42.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:42 smithi083 ceph-mon[32919]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:42.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:42 smithi102 ceph-mon[33005]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:42.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:42 smithi019 ceph-mon[28090]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:44.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:44 smithi083 ceph-mon[32919]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:44.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:44 smithi102 ceph-mon[33005]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:44.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:44 smithi019 ceph-mon[28090]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:45.540 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:45 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:45.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:45 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:45.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:45 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:46.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:46 smithi083 ceph-mon[32919]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:46.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:46 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:46.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:46 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:46.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:46 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:07:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:46 smithi102 ceph-mon[33005]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:46 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:46 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:46.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:46 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:07:46.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:46 smithi019 ceph-mon[28090]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:46.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:46 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:46.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:46 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:46.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:46 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:07:48.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:48 smithi102 ceph-mon[33005]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:48.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:48 smithi019 ceph-mon[28090]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:48.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:48 smithi083 ceph-mon[32919]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:50.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:50 smithi102 ceph-mon[33005]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:50.593 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:50 smithi083 ceph-mon[32919]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:50.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:50 smithi019 ceph-mon[28090]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:52.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:52 smithi102 ceph-mon[33005]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:52.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:52 smithi019 ceph-mon[28090]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:52.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:52 smithi083 ceph-mon[32919]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:54.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:54 smithi019 ceph-mon[28090]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:54.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:54 smithi083 ceph-mon[32919]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:54 smithi102 ceph-mon[33005]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:56.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:56 smithi019 ceph-mon[28090]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:56 smithi083 ceph-mon[32919]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:56 smithi102 ceph-mon[33005]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:58.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:07:58 smithi019 ceph-mon[28090]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:58.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:07:58 smithi083 ceph-mon[32919]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:07:58 smithi102 ceph-mon[33005]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:00.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:00 smithi019 ceph-mon[28090]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:00.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:00 smithi083 ceph-mon[32919]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:00 smithi102 ceph-mon[33005]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:02.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:02 smithi019 ceph-mon[28090]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:02.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:02 smithi083 ceph-mon[32919]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:02 smithi102 ceph-mon[33005]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:04.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:04 smithi019 ceph-mon[28090]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:04.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:04 smithi083 ceph-mon[32919]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:04.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:04 smithi102 ceph-mon[33005]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:06.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:06 smithi019 ceph-mon[28090]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:06 smithi083 ceph-mon[32919]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:06 smithi102 ceph-mon[33005]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:08.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:08 smithi019 ceph-mon[28090]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:08.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:08 smithi083 ceph-mon[32919]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:08 smithi102 ceph-mon[33005]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:10.610 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:10 smithi102 ceph-mon[33005]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:10.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:10 smithi019 ceph-mon[28090]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:10.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:10 smithi083 ceph-mon[32919]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:12.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:12 smithi019 ceph-mon[28090]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:12.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:12 smithi083 ceph-mon[32919]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:12.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:12 smithi102 ceph-mon[33005]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:14 smithi019 ceph-mon[28090]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:14.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:14 smithi083 ceph-mon[32919]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:14 smithi102 ceph-mon[33005]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:16.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:16 smithi019 ceph-mon[28090]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:16 smithi083 ceph-mon[32919]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:16 smithi102 ceph-mon[33005]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:18.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:18 smithi019 ceph-mon[28090]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:18.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:18 smithi083 ceph-mon[32919]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:18 smithi102 ceph-mon[33005]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:20.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:20 smithi083 ceph-mon[32919]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:20.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:20 smithi102 ceph-mon[33005]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:20.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:20 smithi019 ceph-mon[28090]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:22.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:22 smithi083 ceph-mon[32919]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:22 smithi102 ceph-mon[33005]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:22 smithi019 ceph-mon[28090]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:24.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:24 smithi083 ceph-mon[32919]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:24.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:24 smithi102 ceph-mon[33005]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:24 smithi019 ceph-mon[28090]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:26.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:26 smithi083 ceph-mon[32919]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:26 smithi102 ceph-mon[33005]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:26.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:26 smithi019 ceph-mon[28090]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:28.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:28 smithi019 ceph-mon[28090]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:28.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:28 smithi083 ceph-mon[32919]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:28.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:28 smithi102 ceph-mon[33005]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:30.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:30 smithi083 ceph-mon[32919]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:30.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:30 smithi102 ceph-mon[33005]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:30.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:30 smithi019 ceph-mon[28090]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:32.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:32 smithi083 ceph-mon[32919]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:32 smithi102 ceph-mon[33005]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:32 smithi019 ceph-mon[28090]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:34.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:34 smithi083 ceph-mon[32919]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:34 smithi102 ceph-mon[33005]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:34.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:34 smithi019 ceph-mon[28090]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:36.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:36 smithi083 ceph-mon[32919]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:36.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:36 smithi102 ceph-mon[33005]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:36.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:36 smithi019 ceph-mon[28090]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:38.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:38 smithi083 ceph-mon[32919]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:38.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:38 smithi102 ceph-mon[33005]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:38 smithi019 ceph-mon[28090]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:40.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:40 smithi083 ceph-mon[32919]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:40 smithi102 ceph-mon[33005]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:40.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:40 smithi019 ceph-mon[28090]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:42.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:42 smithi083 ceph-mon[32919]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:42 smithi102 ceph-mon[33005]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:42 smithi019 ceph-mon[28090]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:44.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:44 smithi083 ceph-mon[32919]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:44 smithi102 ceph-mon[33005]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:44 smithi019 ceph-mon[28090]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:46.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:46 smithi083 ceph-mon[32919]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:46 smithi102 ceph-mon[33005]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:46 smithi019 ceph-mon[28090]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:47.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:47 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:47.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:47 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:47.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:47 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:48 smithi083 ceph-mon[32919]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:48 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:48.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:48 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:48.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:48 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:08:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:48 smithi102 ceph-mon[33005]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:48 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:48 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:48 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:08:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:48 smithi019 ceph-mon[28090]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:48 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:48 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:48 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:08:50.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:50 smithi083 ceph-mon[32919]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:50 smithi102 ceph-mon[33005]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:50.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:50 smithi019 ceph-mon[28090]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:52.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:52 smithi083 ceph-mon[32919]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:52.810 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:52 smithi102 ceph-mon[33005]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:52.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:52 smithi019 ceph-mon[28090]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:54.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:54 smithi083 ceph-mon[32919]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:54 smithi102 ceph-mon[33005]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:54.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:54 smithi019 ceph-mon[28090]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:56.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:56 smithi083 ceph-mon[32919]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:56 smithi102 ceph-mon[33005]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:56.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:56 smithi019 ceph-mon[28090]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:58.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:08:58 smithi083 ceph-mon[32919]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:08:58 smithi102 ceph-mon[33005]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:58.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:08:58 smithi019 ceph-mon[28090]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:00.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:00 smithi083 ceph-mon[32919]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:00 smithi102 ceph-mon[33005]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:00.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:00 smithi019 ceph-mon[28090]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:02.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:02 smithi083 ceph-mon[32919]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:02 smithi102 ceph-mon[33005]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:02 smithi019 ceph-mon[28090]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:04.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:04 smithi083 ceph-mon[32919]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:04.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:04 smithi102 ceph-mon[33005]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:04.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:04 smithi019 ceph-mon[28090]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:06 smithi083 ceph-mon[32919]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:06 smithi102 ceph-mon[33005]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:06.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:06 smithi019 ceph-mon[28090]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:08.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:08 smithi083 ceph-mon[32919]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:08 smithi102 ceph-mon[33005]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:08.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:08 smithi019 ceph-mon[28090]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:10.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:10 smithi083 ceph-mon[32919]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:10 smithi102 ceph-mon[33005]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:10.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:10 smithi019 ceph-mon[28090]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:12.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:12 smithi083 ceph-mon[32919]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:12.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:12 smithi102 ceph-mon[33005]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:12 smithi019 ceph-mon[28090]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:14.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:14 smithi083 ceph-mon[32919]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:14 smithi102 ceph-mon[33005]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:14.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:14 smithi019 ceph-mon[28090]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:09:16.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:16 smithi083 ceph-mon[32919]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:16 smithi102 ceph-mon[33005]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:16 smithi019 ceph-mon[28090]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:09:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:18 smithi102 ceph-mon[33005]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:18 smithi019 ceph-mon[28090]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:19.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:18 smithi083 ceph-mon[32919]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:20.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:20 smithi102 ceph-mon[33005]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:20.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:20 smithi083 ceph-mon[32919]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:20 smithi019 ceph-mon[28090]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:09:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:22 smithi102 ceph-mon[33005]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:09:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:22 smithi019 ceph-mon[28090]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:09:23.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:22 smithi083 ceph-mon[32919]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:09:24.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:24 smithi102 ceph-mon[33005]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:24 smithi019 ceph-mon[28090]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:25.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:24 smithi083 ceph-mon[32919]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:26.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:26 smithi019 ceph-mon[28090]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:27.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:26 smithi083 ceph-mon[32919]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:27.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:26 smithi102 ceph-mon[33005]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:28.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:28 smithi019 ceph-mon[28090]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:29.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:28 smithi083 ceph-mon[32919]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:29.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:28 smithi102 ceph-mon[33005]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:30.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:30 smithi019 ceph-mon[28090]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:30 smithi083 ceph-mon[32919]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:31.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:30 smithi102 ceph-mon[33005]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:32.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:32 smithi019 ceph-mon[28090]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:33.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:32 smithi083 ceph-mon[32919]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:32 smithi102 ceph-mon[33005]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:34 smithi019 ceph-mon[28090]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:35.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:34 smithi083 ceph-mon[32919]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:35.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:34 smithi102 ceph-mon[33005]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:36 smithi019 ceph-mon[28090]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:37.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:36 smithi083 ceph-mon[32919]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:37.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:36 smithi102 ceph-mon[33005]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:38 smithi019 ceph-mon[28090]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:38 smithi083 ceph-mon[32919]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:39.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:38 smithi102 ceph-mon[33005]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:40.860 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:40 smithi102 ceph-mon[33005]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:40.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:40 smithi019 ceph-mon[28090]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:41.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:40 smithi083 ceph-mon[32919]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:42 smithi019 ceph-mon[28090]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:42 smithi083 ceph-mon[32919]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:43.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:42 smithi102 ceph-mon[33005]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:44.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:44 smithi019 ceph-mon[28090]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:45.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:44 smithi083 ceph-mon[32919]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:45.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:44 smithi102 ceph-mon[33005]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:46 smithi019 ceph-mon[28090]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:47.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:46 smithi083 ceph-mon[32919]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:47.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:46 smithi102 ceph-mon[33005]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:48.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:48 smithi019 ceph-mon[28090]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:48 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:48 smithi083 ceph-mon[32919]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:48 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:48 smithi102 ceph-mon[33005]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:48 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:50 smithi083 ceph-mon[32919]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:50 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:50 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:50 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:09:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:50 smithi102 ceph-mon[33005]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:50 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:50 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:50 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:09:51.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:50 smithi019 ceph-mon[28090]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:50 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:51.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:50 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:51.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:50 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:09:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:52 smithi083 ceph-mon[32919]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:52 smithi102 ceph-mon[33005]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:53.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:52 smithi019 ceph-mon[28090]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:55.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:54 smithi083 ceph-mon[32919]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:54 smithi102 ceph-mon[33005]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:55.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:54 smithi019 ceph-mon[28090]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:56 smithi083 ceph-mon[32919]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:56 smithi102 ceph-mon[33005]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:56 smithi019 ceph-mon[28090]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:58.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:09:58 smithi019 ceph-mon[28090]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:59.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:09:58 smithi083 ceph-mon[32919]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:09:58 smithi102 ceph-mon[33005]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:01.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:00 smithi083 ceph-mon[32919]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:01.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:00 smithi083 ceph-mon[32919]: overall HEALTH_OK 2024-04-25T15:10:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:00 smithi102 ceph-mon[33005]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:00 smithi102 ceph-mon[33005]: overall HEALTH_OK 2024-04-25T15:10:01.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:00 smithi019 ceph-mon[28090]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:01.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:00 smithi019 ceph-mon[28090]: overall HEALTH_OK 2024-04-25T15:10:03.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:02 smithi083 ceph-mon[32919]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:03.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:02 smithi102 ceph-mon[33005]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:03.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:02 smithi019 ceph-mon[28090]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:04 smithi083 ceph-mon[32919]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:05.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:04 smithi102 ceph-mon[33005]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:05.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:04 smithi019 ceph-mon[28090]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:07.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:06 smithi083 ceph-mon[32919]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:07.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:06 smithi102 ceph-mon[33005]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:07.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:06 smithi019 ceph-mon[28090]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:09.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:08 smithi083 ceph-mon[32919]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:09.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:08 smithi102 ceph-mon[33005]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:09.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:08 smithi019 ceph-mon[28090]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:10.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:09 smithi083 ceph-mon[32919]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:10.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:09 smithi102 ceph-mon[33005]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:10.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:09 smithi019 ceph-mon[28090]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:12.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:12 smithi019 ceph-mon[28090]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:12.760 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:12 smithi102 ceph-mon[33005]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:12.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:12 smithi083 ceph-mon[32919]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:14.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:14 smithi019 ceph-mon[28090]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:14.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:14 smithi083 ceph-mon[32919]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:14 smithi102 ceph-mon[33005]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:10:16.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:16 smithi019 ceph-mon[28090]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:16 smithi083 ceph-mon[32919]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:16 smithi102 ceph-mon[33005]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:18.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:18 smithi019 ceph-mon[28090]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:18.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:18 smithi083 ceph-mon[32919]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:18 smithi102 ceph-mon[33005]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:20.593 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:20 smithi083 ceph-mon[32919]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:20.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:20 smithi019 ceph-mon[28090]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:20.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:20 smithi102 ceph-mon[33005]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:10:22.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:22 smithi019 ceph-mon[28090]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:10:22.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:22 smithi083 ceph-mon[32919]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:10:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:22 smithi102 ceph-mon[33005]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:10:24.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:24 smithi019 ceph-mon[28090]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:24.792 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:24 smithi083 ceph-mon[32919]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:24.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:24 smithi102 ceph-mon[33005]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:26.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:26 smithi019 ceph-mon[28090]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:26.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:26 smithi083 ceph-mon[32919]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:26 smithi102 ceph-mon[33005]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:28.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:28 smithi019 ceph-mon[28090]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:28.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:28 smithi083 ceph-mon[32919]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:28.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:28 smithi102 ceph-mon[33005]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:30.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:30 smithi019 ceph-mon[28090]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:30.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:30 smithi083 ceph-mon[32919]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:30.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:30 smithi102 ceph-mon[33005]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:32.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:32 smithi019 ceph-mon[28090]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:32.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:32 smithi083 ceph-mon[32919]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:32 smithi102 ceph-mon[33005]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:34.621 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:34 smithi019 ceph-mon[28090]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:34.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:34 smithi083 ceph-mon[32919]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:34 smithi102 ceph-mon[33005]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:36.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:36 smithi083 ceph-mon[32919]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:36.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:36 smithi102 ceph-mon[33005]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:36 smithi019 ceph-mon[28090]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:38.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:38 smithi083 ceph-mon[32919]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:38.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:38 smithi102 ceph-mon[33005]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:38 smithi019 ceph-mon[28090]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:40.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:40 smithi083 ceph-mon[32919]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:40 smithi102 ceph-mon[33005]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:40.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:40 smithi019 ceph-mon[28090]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:42.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:42 smithi083 ceph-mon[32919]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:42.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:42 smithi102 ceph-mon[33005]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:42 smithi019 ceph-mon[28090]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:44.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:44 smithi083 ceph-mon[32919]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:44.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:44 smithi102 ceph-mon[33005]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:44.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:44 smithi019 ceph-mon[28090]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:46.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:46 smithi083 ceph-mon[32919]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:46.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:46 smithi102 ceph-mon[33005]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:46 smithi019 ceph-mon[28090]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:48.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:48 smithi083 ceph-mon[32919]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:48.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:48 smithi102 ceph-mon[33005]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:48.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:48 smithi019 ceph-mon[28090]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:50.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:50 smithi083 ceph-mon[32919]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:50.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:50 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:50 smithi102 ceph-mon[33005]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:50.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:50 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:50 smithi019 ceph-mon[28090]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:50 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:51.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:51 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:51.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:51 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:51.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:51 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:10:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:51 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:51 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:51.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:51 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:10:51.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:51 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:51.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:51 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:51.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:51 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:10:52.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:52 smithi083 ceph-mon[32919]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:52.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:52 smithi102 ceph-mon[33005]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:52.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:52 smithi019 ceph-mon[28090]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:54.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:54 smithi083 ceph-mon[32919]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:54.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:54 smithi102 ceph-mon[33005]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:54.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:54 smithi019 ceph-mon[28090]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:56.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:56 smithi083 ceph-mon[32919]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:56.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:56 smithi102 ceph-mon[33005]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:56.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:56 smithi019 ceph-mon[28090]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:58.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:10:58 smithi083 ceph-mon[32919]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:58.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:10:58 smithi102 ceph-mon[33005]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:58.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:10:58 smithi019 ceph-mon[28090]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:00.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:00 smithi083 ceph-mon[32919]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:00.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:00 smithi102 ceph-mon[33005]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:00.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:00 smithi019 ceph-mon[28090]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:02.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:02 smithi083 ceph-mon[32919]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:02.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:02 smithi102 ceph-mon[33005]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:02.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:02 smithi019 ceph-mon[28090]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:04.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:04 smithi083 ceph-mon[32919]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:04.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:04 smithi102 ceph-mon[33005]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:04.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:04 smithi019 ceph-mon[28090]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:06.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:06 smithi083 ceph-mon[32919]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:06 smithi102 ceph-mon[33005]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:06.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:06 smithi019 ceph-mon[28090]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:08.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:08 smithi083 ceph-mon[32919]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:08 smithi102 ceph-mon[33005]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:08.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:08 smithi019 ceph-mon[28090]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:09.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:09 smithi083 ceph-mon[32919]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:11:09.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:09 smithi102 ceph-mon[33005]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:11:09.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:09 smithi019 ceph-mon[28090]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:11:10.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:10 smithi083 ceph-mon[32919]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:10 smithi102 ceph-mon[33005]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:10.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:10 smithi019 ceph-mon[28090]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:12.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:12 smithi083 ceph-mon[32919]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:12.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:12 smithi102 ceph-mon[33005]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:12 smithi019 ceph-mon[28090]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:14.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:14 smithi083 ceph-mon[32919]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:14.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:14 smithi083 ceph-mon[32919]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:11:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:14 smithi102 ceph-mon[33005]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:14 smithi102 ceph-mon[33005]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:11:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:14 smithi019 ceph-mon[28090]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:11:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:14 smithi019 ceph-mon[28090]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:11:16.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:16 smithi083 ceph-mon[32919]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:16 smithi102 ceph-mon[33005]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:16 smithi019 ceph-mon[28090]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:18.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:18 smithi083 ceph-mon[32919]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T15:11:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:18 smithi102 ceph-mon[33005]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T15:11:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:18 smithi019 ceph-mon[28090]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-25T15:11:20.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:20 smithi083 ceph-mon[32919]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:20.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:20 smithi102 ceph-mon[33005]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:20 smithi019 ceph-mon[28090]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:22.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:22 smithi083 ceph-mon[32919]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:22 smithi102 ceph-mon[33005]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:22 smithi019 ceph-mon[28090]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:24.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:24 smithi083 ceph-mon[32919]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:24.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:24 smithi102 ceph-mon[33005]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:24.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:24 smithi019 ceph-mon[28090]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:26.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:26 smithi083 ceph-mon[32919]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:11:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:26 smithi102 ceph-mon[33005]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:11:26.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:26 smithi019 ceph-mon[28090]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:11:28.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:28 smithi083 ceph-mon[32919]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:28.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:28 smithi102 ceph-mon[33005]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:28.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:28 smithi019 ceph-mon[28090]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:30.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:30 smithi083 ceph-mon[32919]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:30.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:30 smithi102 ceph-mon[33005]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:30.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:30 smithi019 ceph-mon[28090]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:32.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:32 smithi083 ceph-mon[32919]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:32.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:32 smithi102 ceph-mon[33005]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:32 smithi019 ceph-mon[28090]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:34.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:34 smithi102 ceph-mon[33005]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:34.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:34 smithi019 ceph-mon[28090]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:35.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:34 smithi083 ceph-mon[32919]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:36.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:36 smithi102 ceph-mon[33005]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:36 smithi019 ceph-mon[28090]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:37.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:36 smithi083 ceph-mon[32919]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:38.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:38 smithi102 ceph-mon[33005]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:38 smithi019 ceph-mon[28090]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:38 smithi083 ceph-mon[32919]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:40.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:40 smithi102 ceph-mon[33005]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:40.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:40 smithi019 ceph-mon[28090]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:41.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:40 smithi083 ceph-mon[32919]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:42 smithi019 ceph-mon[28090]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:42 smithi083 ceph-mon[32919]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:43.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:42 smithi102 ceph-mon[33005]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:44 smithi019 ceph-mon[28090]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:45.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:44 smithi083 ceph-mon[32919]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:45.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:44 smithi102 ceph-mon[33005]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:46.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:46 smithi019 ceph-mon[28090]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:47.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:46 smithi083 ceph-mon[32919]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:47.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:46 smithi102 ceph-mon[33005]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:48 smithi019 ceph-mon[28090]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:48 smithi083 ceph-mon[32919]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:48 smithi102 ceph-mon[33005]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:50.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:50 smithi083 ceph-mon[32919]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:50 smithi019 ceph-mon[28090]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:50 smithi102 ceph-mon[33005]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:52.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:52 smithi019 ceph-mon[28090]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:52.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:52 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:52 smithi083 ceph-mon[32919]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:52 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:52 smithi102 ceph-mon[33005]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:52 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:53.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:53 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:53.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:53 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:53.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:53 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:11:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:53 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:53 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:54.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:53 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:11:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:53 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:53 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:53 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:11:54.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:54 smithi019 ceph-mon[28090]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:55.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:54 smithi083 ceph-mon[32919]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:54 smithi102 ceph-mon[33005]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:56.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:56 smithi019 ceph-mon[28090]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:56 smithi083 ceph-mon[32919]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:56 smithi102 ceph-mon[33005]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:58.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:11:58 smithi019 ceph-mon[28090]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:59.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:11:58 smithi083 ceph-mon[32919]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:11:58 smithi102 ceph-mon[33005]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:00.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:00 smithi019 ceph-mon[28090]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:01.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:00 smithi083 ceph-mon[32919]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:00 smithi102 ceph-mon[33005]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:02.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:02 smithi019 ceph-mon[28090]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:12:03.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:02 smithi083 ceph-mon[32919]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:12:03.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:02 smithi102 ceph-mon[33005]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:12:04.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:04 smithi019 ceph-mon[28090]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:12:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:04 smithi083 ceph-mon[32919]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:12:05.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:04 smithi102 ceph-mon[33005]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:12:07.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:06 smithi083 ceph-mon[32919]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:07.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:06 smithi102 ceph-mon[33005]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:07.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:06 smithi019 ceph-mon[28090]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:09.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:08 smithi083 ceph-mon[32919]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:09.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:08 smithi102 ceph-mon[33005]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:09.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:08 smithi019 ceph-mon[28090]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:10.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:09 smithi083 ceph-mon[32919]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:12:10.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:09 smithi102 ceph-mon[33005]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:12:10.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:09 smithi019 ceph-mon[28090]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:12:11.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:10 smithi083 ceph-mon[32919]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:11.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:10 smithi102 ceph-mon[33005]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:11.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:10 smithi019 ceph-mon[28090]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:12 smithi083 ceph-mon[32919]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:12 smithi102 ceph-mon[33005]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:13.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:12 smithi019 ceph-mon[28090]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:14.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:14 smithi083 ceph-mon[32919]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:14.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:14 smithi083 ceph-mon[32919]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:12:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:14 smithi102 ceph-mon[33005]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:14.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:14 smithi102 ceph-mon[33005]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:12:14.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:14 smithi019 ceph-mon[28090]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T15:12:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:14 smithi019 ceph-mon[28090]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:12:16.794 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:16 smithi083 ceph-mon[32919]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:12:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:16 smithi102 ceph-mon[33005]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:12:16.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:16 smithi019 ceph-mon[28090]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:12:18.790 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:18 smithi083 ceph-mon[32919]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:12:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:18 smithi102 ceph-mon[33005]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:12:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:18 smithi019 ceph-mon[28090]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:12:20.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:20 smithi083 ceph-mon[32919]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:12:20.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:20 smithi102 ceph-mon[33005]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:12:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:20 smithi019 ceph-mon[28090]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:12:22.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:22 smithi102 ceph-mon[33005]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:22 smithi019 ceph-mon[28090]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:23.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:22 smithi083 ceph-mon[32919]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:24.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:24 smithi102 ceph-mon[33005]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:24 smithi019 ceph-mon[28090]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:25.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:24 smithi083 ceph-mon[32919]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:12:26.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:26 smithi102 ceph-mon[33005]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:12:26.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:26 smithi019 ceph-mon[28090]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:12:27.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:26 smithi083 ceph-mon[32919]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:12:28.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:28 smithi019 ceph-mon[28090]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:29.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:28 smithi083 ceph-mon[32919]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:29.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:28 smithi102 ceph-mon[33005]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:30.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:30 smithi019 ceph-mon[28090]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:30 smithi083 ceph-mon[32919]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:31.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:30 smithi102 ceph-mon[33005]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:32 smithi019 ceph-mon[28090]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:33.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:32 smithi083 ceph-mon[32919]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:32 smithi102 ceph-mon[33005]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:34 smithi019 ceph-mon[28090]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:35.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:34 smithi083 ceph-mon[32919]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:35.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:34 smithi102 ceph-mon[33005]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:36 smithi019 ceph-mon[28090]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:37.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:36 smithi083 ceph-mon[32919]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:37.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:36 smithi102 ceph-mon[33005]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:38.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:38 smithi019 ceph-mon[28090]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:38 smithi083 ceph-mon[32919]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:39.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:38 smithi102 ceph-mon[33005]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:40.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:40 smithi019 ceph-mon[28090]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:41.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:40 smithi083 ceph-mon[32919]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:40 smithi102 ceph-mon[33005]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:42.859 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:42 smithi102 ceph-mon[33005]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:42 smithi019 ceph-mon[28090]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:43.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:42 smithi083 ceph-mon[32919]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:44 smithi019 ceph-mon[28090]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:44 smithi083 ceph-mon[32919]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:45.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:44 smithi102 ceph-mon[33005]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:46.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:46 smithi019 ceph-mon[28090]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:47.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:46 smithi083 ceph-mon[32919]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:47.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:46 smithi102 ceph-mon[33005]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:48.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:48 smithi019 ceph-mon[28090]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:48 smithi083 ceph-mon[32919]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:48 smithi102 ceph-mon[33005]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:50.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:50 smithi019 ceph-mon[28090]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:50 smithi083 ceph-mon[32919]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:50 smithi102 ceph-mon[33005]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:53.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:52 smithi083 ceph-mon[32919]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:52 smithi102 ceph-mon[33005]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:53.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:52 smithi019 ceph-mon[28090]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:54.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:53 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:54.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:53 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:54.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:53 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:55.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:54 smithi083 ceph-mon[32919]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:54 smithi102 ceph-mon[33005]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:55.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:54 smithi019 ceph-mon[28090]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:55 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:55 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:55 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:12:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:55 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:55 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:55 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:12:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:55 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:56.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:55 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:56.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:55 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:12:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:56 smithi083 ceph-mon[32919]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:56 smithi102 ceph-mon[33005]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:56 smithi019 ceph-mon[28090]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:12:58 smithi083 ceph-mon[32919]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:12:58 smithi102 ceph-mon[33005]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:59.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:12:58 smithi019 ceph-mon[28090]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:01.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:00 smithi083 ceph-mon[32919]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:00 smithi102 ceph-mon[33005]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:01.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:00 smithi019 ceph-mon[28090]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:03.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:02 smithi083 ceph-mon[32919]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:03.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:02 smithi102 ceph-mon[33005]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:03.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:02 smithi019 ceph-mon[28090]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:04 smithi083 ceph-mon[32919]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:05.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:04 smithi102 ceph-mon[33005]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:05.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:04 smithi019 ceph-mon[28090]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:06.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:05 smithi083 ceph-mon[32919]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:13:06.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:05 smithi102 ceph-mon[33005]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:13:06.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:05 smithi019 ceph-mon[28090]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:13:08.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:08 smithi083 ceph-mon[32919]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:08.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:08 smithi102 ceph-mon[33005]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:08.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:08 smithi019 ceph-mon[28090]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:10.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:10 smithi083 ceph-mon[32919]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:10.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:10 smithi083 ceph-mon[32919]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:13:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:10 smithi102 ceph-mon[33005]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:10.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:10 smithi102 ceph-mon[33005]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:13:10.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:10 smithi019 ceph-mon[28090]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:13:10.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:10 smithi019 ceph-mon[28090]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:13:12.791 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:12 smithi083 ceph-mon[32919]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:12.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:12 smithi102 ceph-mon[33005]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:12.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:12 smithi019 ceph-mon[28090]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:14.808 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:14 smithi102 ceph-mon[33005]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:14.861 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:14 smithi102 ceph-mon[33005]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:13:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:14 smithi019 ceph-mon[28090]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:14 smithi019 ceph-mon[28090]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:13:15.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:14 smithi083 ceph-mon[32919]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:13:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:14 smithi083 ceph-mon[32919]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:13:16.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:16 smithi102 ceph-mon[33005]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:16 smithi019 ceph-mon[28090]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:16 smithi083 ceph-mon[32919]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:18.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:18 smithi102 ceph-mon[33005]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:18 smithi019 ceph-mon[28090]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:19.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:18 smithi083 ceph-mon[32919]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:13:20.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:20 smithi083 ceph-mon[32919]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-25T15:13:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:20 smithi019 ceph-mon[28090]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-25T15:13:21.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:20 smithi102 ceph-mon[33005]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-25T15:13:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:22 smithi019 ceph-mon[28090]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:23.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:22 smithi083 ceph-mon[32919]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:22 smithi102 ceph-mon[33005]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:24.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:24 smithi019 ceph-mon[28090]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:25.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:24 smithi083 ceph-mon[32919]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:25.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:24 smithi102 ceph-mon[33005]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:13:26.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:26 smithi019 ceph-mon[28090]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:13:27.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:26 smithi083 ceph-mon[32919]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:13:27.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:26 smithi102 ceph-mon[33005]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:13:28.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:28 smithi019 ceph-mon[28090]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:29.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:28 smithi083 ceph-mon[32919]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:29.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:28 smithi102 ceph-mon[33005]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:30.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:30 smithi019 ceph-mon[28090]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:31.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:30 smithi083 ceph-mon[32919]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:31.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:30 smithi102 ceph-mon[33005]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:32 smithi019 ceph-mon[28090]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:33.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:32 smithi083 ceph-mon[32919]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:32 smithi102 ceph-mon[33005]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:34.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:34 smithi019 ceph-mon[28090]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:35.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:34 smithi083 ceph-mon[32919]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:35.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:34 smithi102 ceph-mon[33005]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:36.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:36 smithi019 ceph-mon[28090]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:37.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:36 smithi083 ceph-mon[32919]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:37.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:36 smithi102 ceph-mon[33005]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:38.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:38 smithi019 ceph-mon[28090]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:39.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:38 smithi083 ceph-mon[32919]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:39.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:38 smithi102 ceph-mon[33005]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:40.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:40 smithi019 ceph-mon[28090]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:41.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:40 smithi083 ceph-mon[32919]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:40 smithi102 ceph-mon[33005]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:42.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:42 smithi019 ceph-mon[28090]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:43.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:42 smithi083 ceph-mon[32919]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:43.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:42 smithi102 ceph-mon[33005]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:44.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:44 smithi019 ceph-mon[28090]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:44 smithi083 ceph-mon[32919]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:45.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:44 smithi102 ceph-mon[33005]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:47.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:46 smithi083 ceph-mon[32919]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:47.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:46 smithi102 ceph-mon[33005]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:47.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:46 smithi019 ceph-mon[28090]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:48 smithi083 ceph-mon[32919]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:48 smithi102 ceph-mon[33005]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:49.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:48 smithi019 ceph-mon[28090]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:50 smithi083 ceph-mon[32919]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:50 smithi102 ceph-mon[33005]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:51.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:50 smithi019 ceph-mon[28090]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:52 smithi083 ceph-mon[32919]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:52 smithi102 ceph-mon[33005]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:53.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:52 smithi019 ceph-mon[28090]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:55.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:54 smithi083 ceph-mon[32919]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:54 smithi102 ceph-mon[33005]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:55.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:54 smithi019 ceph-mon[28090]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:56.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:55 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:13:56.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:55 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:13:56.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:55 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:13:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:56 smithi083 ceph-mon[32919]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:56 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:13:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:56 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:13:57.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:56 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:13:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:56 smithi102 ceph-mon[33005]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:56 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:13:57.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:56 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:13:57.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:56 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:13:57.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:56 smithi019 ceph-mon[28090]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:13:57.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:56 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:13:57.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:56 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:13:57.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:56 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:13:58.925 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:13:58 smithi019 ceph-mon[28090]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:59.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:13:58 smithi083 ceph-mon[32919]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:13:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:13:58 smithi102 ceph-mon[33005]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:01.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:00 smithi083 ceph-mon[32919]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:01.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:00 smithi102 ceph-mon[33005]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:01.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:00 smithi019 ceph-mon[28090]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:03.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:02 smithi083 ceph-mon[32919]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:03.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:02 smithi102 ceph-mon[33005]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:03.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:02 smithi019 ceph-mon[28090]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:04 smithi083 ceph-mon[32919]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:05.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:04 smithi102 ceph-mon[33005]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:05.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:04 smithi019 ceph-mon[28090]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:07.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:06 smithi083 ceph-mon[32919]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:14:07.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:06 smithi102 ceph-mon[33005]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:14:07.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:06 smithi019 ceph-mon[28090]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:14:09.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:08 smithi083 ceph-mon[32919]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:14:09.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:08 smithi102 ceph-mon[33005]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:14:09.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:08 smithi019 ceph-mon[28090]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:14:10.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:09 smithi083 ceph-mon[32919]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:14:10.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:09 smithi102 ceph-mon[33005]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:14:10.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:09 smithi019 ceph-mon[28090]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:14:11.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:10 smithi083 ceph-mon[32919]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:11.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:10 smithi102 ceph-mon[33005]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:11.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:10 smithi019 ceph-mon[28090]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:13.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:12 smithi083 ceph-mon[32919]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:13.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:12 smithi102 ceph-mon[33005]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:13.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:12 smithi019 ceph-mon[28090]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:14 smithi019 ceph-mon[28090]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:14.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:14 smithi019 ceph-mon[28090]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:14:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:14 smithi083 ceph-mon[32919]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:14 smithi083 ceph-mon[32919]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:14:15.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:14 smithi102 ceph-mon[33005]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:14:15.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:14 smithi102 ceph-mon[33005]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:14:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:16 smithi019 ceph-mon[28090]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:14:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:16 smithi083 ceph-mon[32919]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:14:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:16 smithi102 ceph-mon[33005]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:14:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:18 smithi019 ceph-mon[28090]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:14:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:18 smithi083 ceph-mon[32919]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:14:19.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:18 smithi102 ceph-mon[33005]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-25T15:14:20.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:20 smithi083 ceph-mon[32919]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:14:20.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:20 smithi019 ceph-mon[28090]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:14:21.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:20 smithi102 ceph-mon[33005]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:14:22.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:22 smithi019 ceph-mon[28090]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:23.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:22 smithi083 ceph-mon[32919]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:23.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:22 smithi102 ceph-mon[33005]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:24.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:24 smithi019 ceph-mon[28090]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:25.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:24 smithi083 ceph-mon[32919]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:25.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:24 smithi102 ceph-mon[33005]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:14:26.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:26 smithi019 ceph-mon[28090]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:14:27.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:26 smithi083 ceph-mon[32919]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:14:27.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:26 smithi102 ceph-mon[33005]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 86 B/s wr, 2 op/s 2024-04-25T15:14:28.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:28 smithi019 ceph-mon[28090]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:29.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:28 smithi083 ceph-mon[32919]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:29.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:28 smithi102 ceph-mon[33005]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:30.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:30 smithi019 ceph-mon[28090]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:31.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:30 smithi083 ceph-mon[32919]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:31.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:30 smithi102 ceph-mon[33005]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:32.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T15:14:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 -- 172.21.15.19:0/3027209816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 msgr2=0x7f9f4410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3027209816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4410ea00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009900 tx=0x7f9f3402a400 comp rx=0 tx=0).stop 2024-04-25T15:14:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 -- 172.21.15.19:0/3027209816 shutdown_connections 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3027209816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3027209816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3027209816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4410ea00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 -- 172.21.15.19:0/3027209816 >> 172.21.15.19:0/3027209816 conn(0x7f9f440764e0 msgr2=0x7f9f44078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 -- 172.21.15.19:0/3027209816 shutdown_connections 2024-04-25T15:14:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.224+0000 7f9f48c00640 1 -- 172.21.15.19:0/3027209816 wait complete. 2024-04-25T15:14:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.225+0000 7f9f48c00640 1 Processor -- start 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 -- start start 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441187b0 con 0x7f9f4410e620 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44118630 con 0x7f9f44108560 2024-04-25T15:14:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f48c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44118930 con 0x7f9f44108f20 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:52954/0 (socket says 172.21.15.19:52954) 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f41e00640 1 -- 172.21.15.19:0/3308666032 learned_addr learned my addr 172.21.15.19:0/3308666032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.226+0000 7f9f42800640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 -- 172.21.15.19:0/3308666032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 msgr2=0x7f9f441a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 -- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 msgr2=0x7f9f4419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 -- 172.21.15.19:0/3308666032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419f230 con 0x7f9f44108f20 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f41e00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.227+0000 7f9f3be00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441a4e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e9f0 tx=0x7f9f2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.228+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00cd80 con 0x7f9f44108f20 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.228+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f2c0035f0 con 0x7f9f44108f20 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.228+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c010670 con 0x7f9f44108f20 2024-04-25T15:14:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.228+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419f520 con 0x7f9f44108f20 2024-04-25T15:14:32.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.228+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441ab7c0 con 0x7f9f44108f20 2024-04-25T15:14:32.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.229+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4404fce0 con 0x7f9f44108f20 2024-04-25T15:14:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.231+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f2c0037a0 con 0x7f9f44108f20 2024-04-25T15:14:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.232+0000 7f9f3aa00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 0x7f9f280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.232+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f9f2c0d09d0 con 0x7f9f44108f20 2024-04-25T15:14:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.233+0000 7f9f41e00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 0x7f9f280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.233+0000 7f9f41e00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 0x7f9f280a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005ec0 tx=0x7f9f34005a20 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.237+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f2c09d010 con 0x7f9f44108f20 2024-04-25T15:14:32.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.467+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9f44062390 con 0x7f9f280a3e40 2024-04-25T15:14:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.481+0000 7f9f3aa00640 1 -- 172.21.15.19:0/3308666032 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9f44062390 con 0x7f9f280a3e40 2024-04-25T15:14:32.483 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T15:04:30", "first": "2024-04-25T15:05:00", "last": "2024-04-25T15:14:00", "last_pruned": "2024-04-25T15:14:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 msgr2=0x7f9f280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 0x7f9f280a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005ec0 tx=0x7f9f34005a20 comp rx=0 tx=0).stop 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 msgr2=0x7f9f441a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441a4e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e9f0 tx=0x7f9f2c00eeb0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 shutdown_connections 2024-04-25T15:14:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f4410e620 0x7f9f4419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f9f280a3e40 0x7f9f280a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f44108f20 0x7f9f441a4e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 --2- 172.21.15.19:0/3308666032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f44108560 0x7f9f441a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 >> 172.21.15.19:0/3308666032 conn(0x7f9f440764e0 msgr2=0x7f9f44077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 shutdown_connections 2024-04-25T15:14:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.485+0000 7f9f48c00640 1 -- 172.21.15.19:0/3308666032 wait complete. 2024-04-25T15:14:32.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T15:14:32.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.703+0000 7fb44ce00640 1 -- 172.21.15.19:0/1382370215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074ca0 msgr2=0x7fb44810e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.703+0000 7fb44ce00640 1 --2- 172.21.15.19:0/1382370215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074ca0 0x7fb44810e050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb430009be0 tx=0x7fb43002a750 comp rx=0 tx=0).stop 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 -- 172.21.15.19:0/1382370215 shutdown_connections 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 --2- 172.21.15.19:0/1382370215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44810e590 0x7fb4481109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 --2- 172.21.15.19:0/1382370215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074ca0 0x7fb44810e050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 --2- 172.21.15.19:0/1382370215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb448074380 0x7fb448074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 -- 172.21.15.19:0/1382370215 >> 172.21.15.19:0/1382370215 conn(0x7fb4480fd8a0 msgr2=0x7fb4480ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 -- 172.21.15.19:0/1382370215 shutdown_connections 2024-04-25T15:14:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.704+0000 7fb44ce00640 1 -- 172.21.15.19:0/1382370215 wait complete. 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 Processor -- start 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 -- start start 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 0x7fb44819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 0x7fb44819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4481141b0 con 0x7fb448074ca0 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb448114030 con 0x7fb448074380 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.705+0000 7fb44ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb448114330 con 0x7fb44810e590 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 0x7fb44819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb446a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 0x7fb44819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:52970/0 (socket says 172.21.15.19:52970) 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 -- 172.21.15.19:0/2548379573 learned_addr learned my addr 172.21.15.19:0/2548379573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 -- 172.21.15.19:0/2548379573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 msgr2=0x7fb44819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 0x7fb44819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 -- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 msgr2=0x7fb44819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 0x7fb44819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 -- 172.21.15.19:0/2548379573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44819f740 con 0x7fb448074380 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447e00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 0x7fb44819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb446a00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 0x7fb44819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.706+0000 7fb447400640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9c0 tx=0x7fb43800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.707+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43800cde0 con 0x7fb448074380 2024-04-25T15:14:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.707+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4380035f0 con 0x7fb448074380 2024-04-25T15:14:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.707+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438010690 con 0x7fb448074380 2024-04-25T15:14:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.707+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44819fa30 con 0x7fb448074380 2024-04-25T15:14:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.707+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4481a72c0 con 0x7fb448074380 2024-04-25T15:14:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.708+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb448075a30 con 0x7fb448074380 2024-04-25T15:14:32.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.710+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb438004000 con 0x7fb448074380 2024-04-25T15:14:32.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.711+0000 7fb43fe00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 0x7fb4280a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:32.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.711+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fb4380d0b00 con 0x7fb448074380 2024-04-25T15:14:32.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.711+0000 7fb446a00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 0x7fb4280a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.712+0000 7fb446a00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 0x7fb4280a6250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb430000f80 tx=0x7fb430005fd0 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:32.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.716+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb438009510 con 0x7fb448074380 2024-04-25T15:14:32.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:32 smithi019 ceph-mon[28090]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.949+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 --> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb448062390 con 0x7fb4280a3d90 2024-04-25T15:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.965+0000 7fb43fe00640 1 -- 172.21.15.19:0/2548379573 <== mgr.15015 v2:172.21.15.19:6800/974415144 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb448062390 con 0x7fb4280a3d90 2024-04-25T15:14:32.967 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 msgr2=0x7fb4280a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 0x7fb4280a6250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb430000f80 tx=0x7fb430005fd0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 msgr2=0x7fb44819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9c0 tx=0x7fb43800ee80 comp rx=0 tx=0).stop 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 shutdown_connections 2024-04-25T15:14:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fb4280a3d90 0x7fb4280a6250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb44810e590 0x7fb44819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb448074ca0 0x7fb44819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 --2- 172.21.15.19:0/2548379573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb448074380 0x7fb44819a750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.968+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 >> 172.21.15.19:0/2548379573 conn(0x7fb4480fd8a0 msgr2=0x7fb4480fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.969+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 shutdown_connections 2024-04-25T15:14:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:32.969+0000 7fb44ce00640 1 -- 172.21.15.19:0/2548379573 wait complete. 2024-04-25T15:14:32.992 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T15:14:33.003 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:14:33.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:32 smithi083 ceph-mon[32919]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:33.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:32 smithi102 ceph-mon[33005]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 -- 172.21.15.19:0/2463621028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8102190 msgr2=0x7f05a8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 --2- 172.21.15.19:0/2463621028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8102190 0x7f05a8102570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0598009900 tx=0x7f059802a400 comp rx=0 tx=0).stop 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 -- 172.21.15.19:0/2463621028 shutdown_connections 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 --2- 172.21.15.19:0/2463621028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a8107260 0x7f05a810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 --2- 172.21.15.19:0/2463621028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102b40 0x7f05a8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 --2- 172.21.15.19:0/2463621028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8102190 0x7f05a8102570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 -- 172.21.15.19:0/2463621028 >> 172.21.15.19:0/2463621028 conn(0x7f05a8079310 msgr2=0x7f05a8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 -- 172.21.15.19:0/2463621028 shutdown_connections 2024-04-25T15:14:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.214+0000 7f05af600640 1 -- 172.21.15.19:0/2463621028 wait complete. 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.215+0000 7f05af600640 1 Processor -- start 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.215+0000 7f05af600640 1 -- start start 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.215+0000 7f05af600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05af600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8107260 0x7f05a81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05af600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a819a770 0x7f05a819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05af600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8112210 con 0x7f05a819a770 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05af600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8112090 con 0x7f05a8107260 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05af600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8112390 con 0x7f05a8102190 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:50684/0 (socket says 172.21.15.19:50684) 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 -- 172.21.15.19:0/728344259 learned_addr learned my addr 172.21.15.19:0/728344259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05ad600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a819a770 0x7f05a819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f059fe00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8107260 0x7f05a81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 -- 172.21.15.19:0/728344259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8107260 msgr2=0x7f05a81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8107260 0x7f05a81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 -- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a819a770 msgr2=0x7f05a819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a819a770 0x7f05a819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.216+0000 7f05acc00640 1 -- 172.21.15.19:0/728344259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a819b370 con 0x7f05a8102190 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05acc00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0598005ab0 tx=0x7f0598003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059803f070 con 0x7f05a8102190 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0598031b60 con 0x7f05a8102190 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a819b600 con 0x7f05a8102190 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598035030 con 0x7f05a8102190 2024-04-25T15:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.217+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81a7740 con 0x7f05a8102190 2024-04-25T15:14:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.218+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a8103660 con 0x7f05a8102190 2024-04-25T15:14:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.221+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0598042070 con 0x7f05a8102190 2024-04-25T15:14:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.222+0000 7f05a6000640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.222+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f05980f15c0 con 0x7f05a8102190 2024-04-25T15:14:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.222+0000 7f059fe00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.223+0000 7f059fe00640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 0x7f058c0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a819bfc0 tx=0x7f0590005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:33.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.224+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05980bdc00 con 0x7f05a8102190 2024-04-25T15:14:33.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.431+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05a8102b40 con 0x7f05a8102190 2024-04-25T15:14:33.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.433+0000 7f05a6000640 1 -- 172.21.15.19:0/728344259 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23044 (secure 0 0 0) 0x7f05980bd500 con 0x7f05a8102190 2024-04-25T15:14:33.435 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:14:33.435 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":99,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:14:13.558373+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:14:33.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 msgr2=0x7f058c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 0x7f058c0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a819bfc0 tx=0x7f0590005f90 comp rx=0 tx=0).stop 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 msgr2=0x7f05a81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0598005ab0 tx=0x7f0598003c10 comp rx=0 tx=0).stop 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 shutdown_connections 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05a819a770 0x7f05a819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8107260 0x7f05a81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 --2- 172.21.15.19:0/728344259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05a8102190 0x7f05a81a06b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 >> 172.21.15.19:0/728344259 conn(0x7f05a8079310 msgr2=0x7f05a8100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:33.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 shutdown_connections 2024-04-25T15:14:33.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.436+0000 7f05af600640 1 -- 172.21.15.19:0/728344259 wait complete. 2024-04-25T15:14:33.458 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:14:33.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:14:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.684+0000 7f36d6600640 1 -- 172.21.15.19:0/1749759358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 msgr2=0x7f36d0105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.684+0000 7f36d6600640 1 --2- 172.21.15.19:0/1749759358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d0105b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009860 tx=0x7f36b802a3c0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 -- 172.21.15.19:0/1749759358 shutdown_connections 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 --2- 172.21.15.19:0/1749759358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 --2- 172.21.15.19:0/1749759358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d0105b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 --2- 172.21.15.19:0/1749759358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 0x7f36d0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 -- 172.21.15.19:0/1749759358 >> 172.21.15.19:0/1749759358 conn(0x7f36d00fc7d0 msgr2=0x7f36d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 -- 172.21.15.19:0/1749759358 shutdown_connections 2024-04-25T15:14:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.685+0000 7f36d6600640 1 -- 172.21.15.19:0/1749759358 wait complete. 2024-04-25T15:14:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.686+0000 7f36d6600640 1 Processor -- start 2024-04-25T15:14:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.686+0000 7f36d6600640 1 -- start start 2024-04-25T15:14:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 0x7f36d01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d01a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36d0118650 con 0x7f36d01061a0 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36d01184d0 con 0x7f36d0100da0 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f36d01187d0 con 0x7f36d01016c0 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58006/0 (socket says 172.21.15.19:58006) 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d5c00640 1 -- 172.21.15.19:0/1904581730 learned_addr learned my addr 172.21.15.19:0/1904581730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36cfe00640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d01a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.687+0000 7f36d5200640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 0x7f36d01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 -- 172.21.15.19:0/1904581730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 msgr2=0x7f36d01a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d01a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 -- 172.21.15.19:0/1904581730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 msgr2=0x7f36d01a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 0x7f36d01a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 -- 172.21.15.19:0/1904581730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d019f1b0 con 0x7f36d01061a0 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.688+0000 7f36d5c00640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36c800e970 tx=0x7f36c800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.689+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c800ccc0 con 0x7f36d01061a0 2024-04-25T15:14:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.689+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d019f4a0 con 0x7f36d01061a0 2024-04-25T15:14:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.689+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c80054d0 con 0x7f36d01061a0 2024-04-25T15:14:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.689+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d01ab740 con 0x7f36d01061a0 2024-04-25T15:14:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.689+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8018430 con 0x7f36d01061a0 2024-04-25T15:14:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.690+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d0068c90 con 0x7f36d01061a0 2024-04-25T15:14:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.692+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36c8005020 con 0x7f36d01061a0 2024-04-25T15:14:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.694+0000 7f36cd600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 0x7f36b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.695+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f36c80d0c20 con 0x7f36d01061a0 2024-04-25T15:14:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.695+0000 7f36d5200640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 0x7f36b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.695+0000 7f36d5200640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 0x7f36b40a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f36d019fb70 tx=0x7f36c0009500 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.698+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36c809d260 con 0x7f36d01061a0 2024-04-25T15:14:33.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:33 smithi019 ceph-mon[28090]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:33.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:33 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/728344259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:14:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.905+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f36d01ab9d0 con 0x7f36d01061a0 2024-04-25T15:14:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.906+0000 7f36cd600640 1 -- 172.21.15.19:0/1904581730 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f36c809cb60 con 0x7f36d01061a0 2024-04-25T15:14:33.907 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:14:33.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 msgr2=0x7f36b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 0x7f36b40a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f36d019fb70 tx=0x7f36c0009500 comp rx=0 tx=0).stop 2024-04-25T15:14:33.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 msgr2=0x7f36d019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:33.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36c800e970 tx=0x7f36c800ee30 comp rx=0 tx=0).stop 2024-04-25T15:14:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 shutdown_connections 2024-04-25T15:14:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36d01061a0 0x7f36d019ea30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f36b40a3da0 0x7f36b40a6260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36d01016c0 0x7f36d01a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 --2- 172.21.15.19:0/1904581730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d0100da0 0x7f36d01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 >> 172.21.15.19:0/1904581730 conn(0x7f36d00fc7d0 msgr2=0x7f36d00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.908+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 shutdown_connections 2024-04-25T15:14:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:33.909+0000 7f36d6600640 1 -- 172.21.15.19:0/1904581730 wait complete. 2024-04-25T15:14:33.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:14:34.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:33 smithi083 ceph-mon[32919]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:34.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:33 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/728344259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:14:34.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:33 smithi102 ceph-mon[33005]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:34.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:33 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/728344259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 -- 172.21.15.19:0/1189539612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 msgr2=0x7f8aec104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1189539612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec104780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009900 tx=0x7f8adc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 -- 172.21.15.19:0/1189539612 shutdown_connections 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1189539612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec109470 0x7f8aec111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1189539612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec104d50 0x7f8aec108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1189539612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec104780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 -- 172.21.15.19:0/1189539612 >> 172.21.15.19:0/1189539612 conn(0x7f8aec0ffbc0 msgr2=0x7f8aec101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:34.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 -- 172.21.15.19:0/1189539612 shutdown_connections 2024-04-25T15:14:34.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.138+0000 7f8af1c00640 1 -- 172.21.15.19:0/1189539612 wait complete. 2024-04-25T15:14:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.139+0000 7f8af1c00640 1 Processor -- start 2024-04-25T15:14:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.139+0000 7f8af1c00640 1 -- start start 2024-04-25T15:14:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 0x7f8aec1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec19a840 0x7f8aec19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec114400 con 0x7f8aec109470 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec114280 con 0x7f8aec1043a0 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8af1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec114580 con 0x7f8aec19a840 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8ae2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 0x7f8aec1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:51014/0 (socket says 172.21.15.19:51014) 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 -- 172.21.15.19:0/1121413486 learned_addr learned my addr 172.21.15.19:0/1121413486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeb400640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec19a840 0x7f8aec19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 -- 172.21.15.19:0/1121413486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec19a840 msgr2=0x7f8aec19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec19a840 0x7f8aec19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 -- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 msgr2=0x7f8aec1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 0x7f8aec1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8aeaa00640 1 -- 172.21.15.19:0/1121413486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec19b3d0 con 0x7f8aec1043a0 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.140+0000 7f8ae2000640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 0x7f8aec1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8aeaa00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8adc005ab0 tx=0x7f8adc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc03f070 con 0x7f8aec1043a0 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8adc031b60 con 0x7f8aec1043a0 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc035030 con 0x7f8aec1043a0 2024-04-25T15:14:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec19b660 con 0x7f8aec1043a0 2024-04-25T15:14:34.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.141+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1a7770 con 0x7f8aec1043a0 2024-04-25T15:14:34.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.142+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec105870 con 0x7f8aec1043a0 2024-04-25T15:14:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.144+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8adc042070 con 0x7f8aec1043a0 2024-04-25T15:14:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.146+0000 7f8ae3e00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 0x7f8ad00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.146+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f8adc0f16c0 con 0x7f8aec1043a0 2024-04-25T15:14:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.146+0000 7f8ae2000640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 0x7f8ad00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.147+0000 7f8ae2000640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 0x7f8ad00a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40095f0 tx=0x7f8ad4005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:34.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.148+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8adc0bd6b0 con 0x7f8aec1043a0 2024-04-25T15:14:34.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.355+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8aec10c700 con 0x7f8aec1043a0 2024-04-25T15:14:34.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.356+0000 7f8ae3e00640 1 -- 172.21.15.19:0/1121413486 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f8adc0c19b0 con 0x7f8aec1043a0 2024-04-25T15:14:34.357 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:14:34.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.358+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 msgr2=0x7f8ad00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.358+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 0x7f8ad00a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40095f0 tx=0x7f8ad4005f90 comp rx=0 tx=0).stop 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.358+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 msgr2=0x7f8aec1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.358+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8adc005ab0 tx=0x7f8adc003c10 comp rx=0 tx=0).stop 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 shutdown_connections 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8ad00a3d20 0x7f8ad00a61e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aec19a840 0x7f8aec19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8aec109470 0x7f8aec1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 --2- 172.21.15.19:0/1121413486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aec1043a0 0x7f8aec1a0680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 >> 172.21.15.19:0/1121413486 conn(0x7f8aec0ffbc0 msgr2=0x7f8aec100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:34.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 shutdown_connections 2024-04-25T15:14:34.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.359+0000 7f8af1c00640 1 -- 172.21.15.19:0/1121413486 wait complete. 2024-04-25T15:14:34.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:14:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.597+0000 7fbdffc00640 1 -- 172.21.15.19:0/3157838830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108f40 msgr2=0x7fbdf81153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.597+0000 7fbdffc00640 1 --2- 172.21.15.19:0/3157838830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108f40 0x7fbdf81153b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009890 tx=0x7fbdf002a370 comp rx=0 tx=0).stop 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 -- 172.21.15.19:0/3157838830 shutdown_connections 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 --2- 172.21.15.19:0/3157838830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108f40 0x7fbdf81153b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 --2- 172.21.15.19:0/3157838830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108580 0x7fbdf8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 --2- 172.21.15.19:0/3157838830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 -- 172.21.15.19:0/3157838830 >> 172.21.15.19:0/3157838830 conn(0x7fbdf8076490 msgr2=0x7fbdf80788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 -- 172.21.15.19:0/3157838830 shutdown_connections 2024-04-25T15:14:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.598+0000 7fbdffc00640 1 -- 172.21.15.19:0/3157838830 wait complete. 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 Processor -- start 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 -- start start 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 0x7fbdf81a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 0x7fbdf81a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf8118760 con 0x7fbdf8108f40 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf81185e0 con 0x7fbdf810e5f0 2024-04-25T15:14:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.599+0000 7fbdffc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf81188e0 con 0x7fbdf8108580 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 0x7fbdf81a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:51030/0 (socket says 172.21.15.19:51030) 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 -- 172.21.15.19:0/2776480546 learned_addr learned my addr 172.21.15.19:0/2776480546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdf7e00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 0x7fbdf81a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 -- 172.21.15.19:0/2776480546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 msgr2=0x7fbdf81a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 0x7fbdf81a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 -- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 msgr2=0x7fbdf81a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 0x7fbdf81a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfdc00640 1 -- 172.21.15.19:0/2776480546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf819f1e0 con 0x7fbdf810e5f0 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdfd200640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 0x7fbdf81a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.600+0000 7fbdf7e00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 0x7fbdf81a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdfdc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0035040 tx=0x7fbdf0030b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0040020 con 0x7fbdf810e5f0 2024-04-25T15:14:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf0038040 con 0x7fbdf810e5f0 2024-04-25T15:14:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf819f470 con 0x7fbdf810e5f0 2024-04-25T15:14:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0037070 con 0x7fbdf810e5f0 2024-04-25T15:14:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.601+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf81ab770 con 0x7fbdf810e5f0 2024-04-25T15:14:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.602+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf80691c0 con 0x7fbdf810e5f0 2024-04-25T15:14:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.604+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdf0043020 con 0x7fbdf810e5f0 2024-04-25T15:14:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.606+0000 7fbdf5600640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.606+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fbdf00f0830 con 0x7fbdf810e5f0 2024-04-25T15:14:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.606+0000 7fbdfd200640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:34.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.607+0000 7fbdfd200640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdf819fba0 tx=0x7fbde80095c0 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.608+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdf004a070 con 0x7fbdf810e5f0 2024-04-25T15:14:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.815+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbdf81109a0 con 0x7fbdf810e5f0 2024-04-25T15:14:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.816+0000 7fbdf5600640 1 -- 172.21.15.19:0/2776480546 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fbdf00bcf80 con 0x7fbdf810e5f0 2024-04-25T15:14:34.817 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:14:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 msgr2=0x7fbddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdf819fba0 tx=0x7fbde80095c0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 msgr2=0x7fbdf819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0035040 tx=0x7fbdf0030b30 comp rx=0 tx=0).stop 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 shutdown_connections 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdf810e5f0 0x7fbdf819ea60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf8108f40 0x7fbdf81a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbddc0a3d20 0x7fbddc0a61e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 --2- 172.21.15.19:0/2776480546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdf8108580 0x7fbdf81a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 >> 172.21.15.19:0/2776480546 conn(0x7fbdf8076490 msgr2=0x7fbdf8077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:34.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 shutdown_connections 2024-04-25T15:14:34.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:34.819+0000 7fbdffc00640 1 -- 172.21.15.19:0/2776480546 wait complete. 2024-04-25T15:14:34.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:34 smithi019 ceph-mon[28090]: from='client.25334 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:34.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:34 smithi019 ceph-mon[28090]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:34.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:34 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1904581730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:14:34.837 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:34 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1121413486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:14:34.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T15:14:35.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:34 smithi083 ceph-mon[32919]: from='client.25334 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:35.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:34 smithi083 ceph-mon[32919]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:35.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:34 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1904581730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:14:35.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:34 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1121413486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:14:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 -- 172.21.15.19:0/525186552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc104d40 msgr2=0x7f21fc108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 --2- 172.21.15.19:0/525186552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc104d40 0x7f21fc108d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009900 tx=0x7f21e402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 -- 172.21.15.19:0/525186552 shutdown_connections 2024-04-25T15:14:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 --2- 172.21.15.19:0/525186552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc109460 0x7f21fc111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 --2- 172.21.15.19:0/525186552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc104d40 0x7f21fc108d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 --2- 172.21.15.19:0/525186552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc104390 0x7f21fc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 -- 172.21.15.19:0/525186552 >> 172.21.15.19:0/525186552 conn(0x7f21fc0ffc50 msgr2=0x7f21fc102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 -- 172.21.15.19:0/525186552 shutdown_connections 2024-04-25T15:14:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.052+0000 7f2202a00640 1 -- 172.21.15.19:0/525186552 wait complete. 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.053+0000 7f2202a00640 1 Processor -- start 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.053+0000 7f2202a00640 1 -- start start 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc104390 0x7f21fc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 0x7f21fc19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 0x7f21fc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1143d0 con 0x7f21fc109460 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc114250 con 0x7f21fc19f3a0 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2202a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc114550 con 0x7f21fc104390 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f21fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 0x7f21fc19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f21fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 0x7f21fc19aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38588/0 (socket says 172.21.15.19:38588) 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f21fb400640 1 -- 172.21.15.19:0/1093259170 learned_addr learned my addr 172.21.15.19:0/1093259170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f2200a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 0x7f21fc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.054+0000 7f21fbe00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc104390 0x7f21fc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 -- 172.21.15.19:0/1093259170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc104390 msgr2=0x7f21fc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:35.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc104390 0x7f21fc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 -- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 msgr2=0x7f21fc19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 0x7f21fc19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 -- 172.21.15.19:0/1093259170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc19fe90 con 0x7f21fc19f3a0 2024-04-25T15:14:35.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:34 smithi102 ceph-mon[33005]: from='client.25334 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:14:35.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:34 smithi102 ceph-mon[33005]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:35.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:34 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1904581730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:14:35.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:34 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1121413486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:14:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2200a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 0x7f21fc19f850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009fd0 tx=0x7f21f400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f400ce20 con 0x7f21fc19f3a0 2024-04-25T15:14:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f4005020 con 0x7f21fc19f3a0 2024-04-25T15:14:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4010690 con 0x7f21fc19f3a0 2024-04-25T15:14:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.055+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1a0180 con 0x7f21fc19f3a0 2024-04-25T15:14:35.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.056+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a7940 con 0x7f21fc19f3a0 2024-04-25T15:14:35.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.057+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0691c0 con 0x7f21fc19f3a0 2024-04-25T15:14:35.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.061+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f21f40054d0 con 0x7f21fc19f3a0 2024-04-25T15:14:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.062+0000 7f21f8c00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 0x7f21dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:14:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.062+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f21f40d0e50 con 0x7f21fc19f3a0 2024-04-25T15:14:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.062+0000 7f21fbe00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 0x7f21dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:14:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.063+0000 7f21fbe00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 0x7f21dc0a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009bd0 tx=0x7f21ec005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:14:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.067+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21f409d490 con 0x7f21fc19f3a0 2024-04-25T15:14:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.327+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f21fc10c690 con 0x7f21fc19f3a0 2024-04-25T15:14:35.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.338+0000 7f21f8c00640 1 -- 172.21.15.19:0/1093259170 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746060 (secure 0 0 0) 0x7f21f409cd90 con 0x7f21fc19f3a0 2024-04-25T15:14:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:14:35.354 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:14:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-04-25T15:14:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 15015, 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-04-25T15:14:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 974415144 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 974415144 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/974415144", 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-04-25T15:04:29.064980+0000", 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 25130, 2024-04-25T15:14:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T15:14:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:14:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T15:14:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T15:14:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T15:14:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:14:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T15:14:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T15:14:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T15:14:35.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:35.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T15:14:35.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.417 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T15:14:35.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T15:14:35.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.431 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T15:14:35.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:14:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T15:14:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T15:14:35.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T15:14:35.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T15:14:35.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T15:14:35.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T15:14:35.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:14:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:14:35.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T15:14:35.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:14:35.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:14:35.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.476 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.481 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:14:35.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.489 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:14:35.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T15:14:35.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T15:14:35.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:14:35.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.506 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T15:14:35.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:14:35.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T15:14:35.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.518 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T15:14:35.527 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.529 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.530 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.533 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T15:14:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.540 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:14:35.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:14:35.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:14:35.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:14:35.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.562 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.567 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.574 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:14:35.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T15:14:35.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:14:35.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.593 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.595 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:14:35.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:14:35.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:14:35.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T15:14:35.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.609 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:14:35.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.616 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:14:35.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.625 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T15:14:35.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:14:35.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.633 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.638 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T15:14:35.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:14:35.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.648 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:35.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.653 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:35.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T15:14:35.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.662 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T15:14:35.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T15:14:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:14:35.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T15:14:35.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T15:14:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.683 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T15:14:35.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T15:14:35.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:14:35.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.705 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.709 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T15:14:35.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:35.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.721 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.725 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.728 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.730 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.735 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.736 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.739 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.742 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.743 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T15:14:35.745 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T15:14:35.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.750 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.755 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.766 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T15:14:35.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:14:35.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.776 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.780 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.783 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T15:14:35.788 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.790 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.791 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.794 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.798 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.807 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T15:14:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.825 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.829 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.835 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T15:14:35.838 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.840 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.841 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.843 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.845 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.848 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T15:14:35.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T15:14:35.852 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.858 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.862 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:35.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.875 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.879 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T15:14:35.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:35.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:14:35.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T15:14:35.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.903 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.909 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T15:14:35.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T15:14:35.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.911 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:35.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.917 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.918 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.921 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:35.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.930 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.934 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.935 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T15:14:35.938 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.940 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.944 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.947 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.949 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.954 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.960 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T15:14:35.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T15:14:35.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T15:14:35.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.980 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.982 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.990 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:35.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:35.995 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:35.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:35.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:35.999 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.009 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.010 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T15:14:36.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.017 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.022 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.025 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T15:14:36.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T15:14:36.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T15:14:36.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.049 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.050 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.053 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.056 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T15:14:36.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.062 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.068 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.069 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.073 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.077 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T15:14:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.087 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T15:14:36.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T15:14:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.107 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 25147, 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.112 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.116 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.122 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T15:14:36.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.129 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T15:14:36.131 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T15:14:36.132 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.135 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T15:14:36.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:14:36.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.142 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.144 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:36.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.149 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.153 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.154 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T15:14:36.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:14:36.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T15:14:36.159 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.160 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:36.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:14:36.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T15:14:36.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T15:14:36.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:14:36.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.175 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:14:36.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:14:36.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T15:14:36.180 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T15:14:36.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.186 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T15:14:36.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:14:36.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:14:36.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.200 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.203 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:14:36.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T15:14:36.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:14:36.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.212 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T15:14:36.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:14:36.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T15:14:36.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:14:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T15:14:36.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T15:14:36.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:14:36.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.234 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.235 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:14:36.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:14:36.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:14:36.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.249 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:14:36.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:14:36.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T15:14:36.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:14:36.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T15:14:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T15:14:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:14:36.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T15:14:36.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T15:14:36.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:14:36.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.260 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T15:14:36.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T15:14:36.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T15:14:36.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.264 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:36.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:36.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T15:14:36.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.266 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T15:14:36.266 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T15:14:36.267 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:14:36.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.268 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T15:14:36.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T15:14:36.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T15:14:36.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:14:36.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.272 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T15:14:36.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:14:36.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:14:36.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:14:36.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.276 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T15:14:36.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T15:14:36.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:14:36.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.280 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T15:14:36.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T15:14:36.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:14:36.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.284 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T15:14:36.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T15:14:36.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:14:36.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T15:14:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T15:14:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:14:36.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T15:14:36.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:14:36.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.296 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T15:14:36.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T15:14:36.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:14:36.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.299 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:14:36.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.302 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T15:14:36.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:14:36.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:14:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.306 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T15:14:36.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:14:36.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:14:36.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T15:14:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:14:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.319 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T15:14:36.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:14:36.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.333 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:14:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:14:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.336 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:14:36.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:14:36.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:14:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:14:36.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.342 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:14:36.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:14:36.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.351 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:14:36.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.359 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:14:36.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.365 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:14:36.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:14:36.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:14:36.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.378 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:14:36.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:14:36.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:14:36.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:14:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:14:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T15:14:36.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.403 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:14:36.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:14:36.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T15:14:36.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.406 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:14:36.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:14:36.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:14:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:14:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:36.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.412 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:14:36.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:14:36.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:14:36.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:14:36.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.427 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T15:14:36.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T15:14:36.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T15:14:36.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.433 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.437 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.442 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.443 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.451 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:36.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T15:14:36.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:36.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T15:14:36.463 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.467 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.472 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T15:14:36.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:14:36.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:14:36.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T15:14:36.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:36.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T15:14:36.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.488 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.490 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.494 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.501 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T15:14:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T15:14:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T15:14:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:14:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T15:14:36.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:14:36.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T15:14:36.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.519 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.528 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:14:36.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.531 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T15:14:36.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T15:14:36.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T15:14:36.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T15:14:36.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:36.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:36.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T15:14:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:36.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.537 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T15:14:36.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T15:14:36.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:36.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:36.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T15:14:36.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T15:14:36.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T15:14:36.544 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.545 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.545 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.545 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.550 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.553 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.553 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.553 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.554 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.554 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.554 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.556 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.562 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.566 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:14:36.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:14:36.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.575 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.582 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.583 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.586 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T15:14:36.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T15:14:36.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.605 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.609 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.614 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.624 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.632 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.634 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.640 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.646 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.655 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.661 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.662 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.664 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.666 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.675 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.677 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.681 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.687 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.694 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T15:14:36.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:14:36.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.699 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T15:14:36.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T15:14:36.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.711 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T15:14:36.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.716 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:36.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.722 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:14:36.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:36.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.741 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.753 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.755 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.760 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.765 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.774 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.780 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.782 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.787 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T15:14:36.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:36.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.811 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T15:14:36.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T15:14:36.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.819 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.824 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.828 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.830 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.833 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.835 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.836 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.841 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.842 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.855 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.860 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.864 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.874 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T15:14:36.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T15:14:36.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.882 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T15:14:36.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T15:14:36.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.896 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:14:36.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.906 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.907 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.908 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.916 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.917 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.922 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.924 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T15:14:36.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.935 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.936 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.938 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.940 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.942 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.944 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.945 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.947 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.949 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T15:14:36.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T15:14:36.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T15:14:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-04-25T15:14:36.959 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T15:14:36.960 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.964 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.965 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:36.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:36.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.973 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:36.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:36.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T15:14:36.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T15:14:36.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T15:14:36.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T15:14:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:36.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T15:14:36.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:36.991 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:36.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:36.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T15:14:37.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:37.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.009 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.014 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T15:14:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T15:14:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T15:14:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T15:14:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T15:14:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:14:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:14:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T15:14:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.047 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T15:14:37.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T15:14:37.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:14:37.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T15:14:37.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:14:37.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T15:14:37.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:14:37.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:14:37.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:14:37.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:37.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.100 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.102 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.106 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T15:14:37.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.112 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T15:14:37.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T15:14:37.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.129 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.131 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T15:14:37.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T15:14:37.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T15:14:37.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:14:37.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.155 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.162 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T15:14:37.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T15:14:37.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:14:37.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.183 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:14:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:14:37.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:14:37.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:37.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.200 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:14:37.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.203 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:14:37.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:14:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:14:37.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.225 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.227 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.229 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.231 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:14:37.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:14:37.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:14:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:14:37.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.248 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.250 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:14:37.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:14:37.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:14:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.260 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.273 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T15:14:37.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:37.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:37.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T15:14:37.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:37.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.296 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.300 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T15:14:37.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:14:37.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:14:37.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.322 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.325 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.327 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.329 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T15:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T15:14:37.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T15:14:37.338 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T15:14:37.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.346 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.348 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T15:14:37.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T15:14:37.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T15:14:37.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T15:14:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T15:14:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:14:37.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.369 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.374 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.377 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.378 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.380 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.384 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T15:14:37.388 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.399 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:14:37.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.406 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T15:14:37.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T15:14:37.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T15:14:37.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:14:37.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:14:37.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.424 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.431 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.436 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.445 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.447 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.452 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.453 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T15:14:37.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.459 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.464 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.468 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T15:14:37.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T15:14:37.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T15:14:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.470 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.473 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.488 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.492 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.494 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.498 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T15:14:37.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.517 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T15:14:37.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T15:14:37.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:14:37.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T15:14:37.524 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:14:37.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.530 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.535 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.536 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T15:14:37.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:14:37.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T15:14:37.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T15:14:37.554 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.560 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.561 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.572 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.574 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.577 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.582 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.591 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.594 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.598 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T15:14:37.602 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.604 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.609 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T15:14:37.615 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.622 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T15:14:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T15:14:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.628 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.630 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T15:14:37.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T15:14:37.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T15:14:37.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T15:14:37.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T15:14:37.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T15:14:37.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T15:14:37.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T15:14:37.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.658 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.660 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.663 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.665 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.675 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.681 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T15:14:37.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:37.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.696 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.704 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T15:14:37.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.708 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T15:14:37.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:14:37.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T15:14:37.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:37.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T15:14:37.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.735 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.741 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.742 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:14:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:14:37.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T15:14:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T15:14:37.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.773 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T15:14:37.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T15:14:37.777 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T15:14:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.780 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T15:14:37.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T15:14:37.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T15:14:37.785 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.786 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T15:14:37.787 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T15:14:37.788 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.789 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T15:14:37.790 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T15:14:37.791 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T15:14:37.793 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T15:14:37.793 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.793 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-04-25T15:14:37.793 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T15:14:37.794 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T15:14:37.795 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 91, 2024-04-25T15:14:37.796 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2276213291 2024-04-25T15:14:37.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.798 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1370289169 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 4178403574 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T15:14:37.800 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2598622017 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1465181598 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2431366332 2024-04-25T15:14:37.803 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.804 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:37.804 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T15:14:37.804 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:14:37.805 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:35 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2776480546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:14:37.806 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:35 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1093259170' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:14:37.806 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:36 smithi102 ceph-mon[33005]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:37.806 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:35 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2776480546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:14:37.806 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:35 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1093259170' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:14:37.806 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:36 smithi083 ceph-mon[32919]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 msgr2=0x7f21dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 0x7f21dc0a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009bd0 tx=0x7f21ec005f90 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 msgr2=0x7f21fc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 0x7f21fc19f850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009fd0 tx=0x7f21f400ef10 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 shutdown_connections 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21fc19f3a0 0x7f21fc19f850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc109460 0x7f21fc19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f21dc0a3da0 0x7f21dc0a6260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 --2- 172.21.15.19:0/1093259170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21fc104390 0x7f21fc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:14:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.353+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 >> 172.21.15.19:0/1093259170 conn(0x7f21fc0ffc50 msgr2=0x7f21fc100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:14:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.354+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 shutdown_connections 2024-04-25T15:14:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:14:35.354+0000 7f2202a00640 1 -- 172.21.15.19:0/1093259170 wait complete. 2024-04-25T15:14:37.815 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T15:14:37.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T15:14:37.816 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:35 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2776480546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:14:37.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:35 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1093259170' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:14:37.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:36 smithi019 ceph-mon[28090]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:14:38.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:38 smithi019 ceph-mon[28090]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:39.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:38 smithi083 ceph-mon[32919]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:39.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:38 smithi102 ceph-mon[33005]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:40.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:40 smithi019 ceph-mon[28090]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:40.842 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:14:40.842 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1560, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1562, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 554421, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 496090, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.324947085, 2024-04-25T15:14:40.843 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.118722023, 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.082195935, 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.086957510, 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1560, 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.135368927, 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000086774 2024-04-25T15:14:40.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.845 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 10, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 164, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 4920, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 446538, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-25T15:14:40.846 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.038566154, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.024353154, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.008605355, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000166222, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 162, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.035492027, 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000219086 2024-04-25T15:14:40.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 445200 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.848 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 977, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 965, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1790693, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 365831, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 3, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.257081942, 2024-04-25T15:14:40.849 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.078976805, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.124151075, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.061988260, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 964, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.090298961, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000093671 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.850 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1608336, 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 130192 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.851 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3842, 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.214161870, 2024-04-25T15:14:40.852 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000055742 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.853 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.854 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-04-25T15:14:40.855 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 12, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 12, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 4, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T15:14:40.856 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T15:14:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T15:14:40.858 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-04-25T15:14:40.859 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 6, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T15:14:40.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-04-25T15:14:40.861 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T15:14:40.862 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T15:14:40.865 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-04-25T15:14:40.866 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-04-25T15:14:40.867 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.868 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 101, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-04-25T15:14:40.869 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 3, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4260878, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4284314, 2024-04-25T15:14:40.870 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4284202, 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 101 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.871 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 13, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 13, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 8, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 8, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 12, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 12, 2024-04-25T15:14:40.872 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 49384, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215416 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T15:14:40.873 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.874 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T15:14:40.875 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.876 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.877 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.878 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T15:14:40.880 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.881 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.881 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.881 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T15:14:40.881 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.882 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.882 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.882 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T15:14:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.889 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T15:14:40.892 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.893 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 5, 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 3, 2024-04-25T15:14:40.894 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 4, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 3188, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:40.896 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:40.897 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:40.898 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:40.899 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 190424, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 52, 2024-04-25T15:14:40.900 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:40.901 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8032, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 167, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 47746, 2024-04-25T15:14:40.902 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 220, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T15:14:40.903 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2305, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 26, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 4, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2305, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2305, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.780982597, 2024-04-25T15:14:40.904 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000772660 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2305, 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2306 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2305, 2024-04-25T15:14:40.905 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2305, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1495, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 807, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-04-25T15:14:40.906 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T15:14:40.907 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 2, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T15:14:40.908 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T15:14:40.909 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T15:14:40.910 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 99, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 11, 2024-04-25T15:14:40.911 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 5, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 5, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.912 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.913 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T15:14:40.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2543, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 2106163, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:40.915 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2543, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2543, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2106163, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T15:14:40.917 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2305, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 90020, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2305, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 90020, 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:40.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T15:14:40.919 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2305, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2305, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2305, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2305, 2024-04-25T15:14:40.920 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T15:14:40.921 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:40.922 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:40.923 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55614c538780": { 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:40.924 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:14:41.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:40 smithi083 ceph-mon[32919]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:41.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:40 smithi102 ceph-mon[33005]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:41.862 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-04-25T15:14:41.874 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T15:14:41.874 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T15:14:41.874 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T15:14:41.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T15:14:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:14:42.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:42 smithi019 ceph-mon[28090]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:43.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:42 smithi083 ceph-mon[32919]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:43.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:42 smithi102 ceph-mon[33005]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:44.445 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:14:44.445 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:44.445 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.003963389, 2024-04-25T15:14:44.446 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000848603, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.448 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 165, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3264, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 185120, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.035724603, 2024-04-25T15:14:44.449 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.026327421, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003994425, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 163, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.029975282, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000183897 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 185120 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 225, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 214, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1590464, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 131744, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.102058918, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.033964988, 2024-04-25T15:14:44.452 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.081478683, 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010413748, 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 213, 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.021915050, 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000102887 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:44.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1590464, 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 131744 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T15:14:44.454 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:44.455 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:44.456 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:44.457 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:44.458 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 149745, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-25T15:14:44.459 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:44.460 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T15:14:44.461 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T15:14:44.462 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 227, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1558750, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 227, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 227, 2024-04-25T15:14:44.463 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1558750, 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:44.464 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:14:44.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:44 smithi019 ceph-mon[28090]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:45.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:44 smithi083 ceph-mon[32919]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:45.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:44 smithi102 ceph-mon[33005]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:45.408 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T15:14:45.408 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T15:14:45.408 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T15:14:45.408 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T15:14:45.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T15:14:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:14:47.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:46 smithi019 ceph-mon[28090]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:47.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:46 smithi083 ceph-mon[32919]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:47.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:46 smithi102 ceph-mon[33005]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:14:47.991 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:14:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 221, 2024-04-25T15:14:47.991 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 210, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1589376, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 129184, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.107651845, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.032922161, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.090646691, 2024-04-25T15:14:47.992 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.009842154, 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 209, 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.021351814, 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000102161 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:47.993 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1589376, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 129184 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 160, 2024-04-25T15:14:47.994 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 184144, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.037744944, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.026165046, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.005631180, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:47.995 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 157, 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.024088098, 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000153427 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:47.996 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 184144 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:14:47.997 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001184339, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000065032, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001077786, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:47.998 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001353674, 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000451224 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:47.999 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:48.001 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:48.002 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:48.003 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:48.004 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148977, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 30, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:14:48.005 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T15:14:48.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T15:14:48.007 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T15:14:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 223, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1558230, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 223, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:48.009 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 223, 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1558230, 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:48.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:48.011 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:14:48.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:48 smithi019 ceph-mon[28090]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:48.944 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T15:14:48.944 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T15:14:48.944 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T15:14:48.944 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T15:14:48.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T15:14:49.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:48 smithi083 ceph-mon[32919]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:49.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:48 smithi102 ceph-mon[33005]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:14:50.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:50 smithi019 ceph-mon[28090]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:51.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:50 smithi083 ceph-mon[32919]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:51.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:50 smithi102 ceph-mon[33005]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:51.799 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-04-25T15:14:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.004898602, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000085705, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001544882, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-04-25T15:14:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001814288, 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000604762 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 227, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 216, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1591008, 2024-04-25T15:14:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 133024, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.099199215, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.024739171, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.091336755, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.010884817, 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 215, 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.021524894, 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000100115 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1591008, 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 133024 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 165, 2024-04-25T15:14:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 185024, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.036643426, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.026263841, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.004556062, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 162, 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.032004997, 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000197561 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T15:14:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 149745, 2024-04-25T15:14:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T15:14:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T15:14:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 229, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1559010, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 229, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T15:14:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 229, 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1559010, 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T15:14:51.820 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T15:14:52.733 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T15:14:52.734 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T15:14:52.734 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T15:14:52.734 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T15:14:52.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T15:14:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:14:53.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:52 smithi083 ceph-mon[32919]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:53.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:52 smithi102 ceph-mon[33005]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:53.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:52 smithi019 ceph-mon[28090]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:54.639 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T15:14:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1201, 2024-04-25T15:14:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 624, 2024-04-25T15:14:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1986910, 2024-04-25T15:14:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 327605, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 11, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.151581612, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.038392955, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.073907491, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.046544161, 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 623, 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.309321122, 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000496502 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 838, 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000168483, 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000201 2024-04-25T15:14:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1932368, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 226032 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 399, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 256, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 141520, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 217542, 2024-04-25T15:14:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 10, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.033505974, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.007889703, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.011312387, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.012453219, 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 254, 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.034821368, 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000137092 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 206, 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000035501, 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000172 2024-04-25T15:14:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 89392, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 144912 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1035, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 497, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 387527, 2024-04-25T15:14:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 588431, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 13, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.137400921, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.033859671, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.044544996, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.054528005, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 495, 2024-04-25T15:14:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.084114518, 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000169928 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 862, 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000155935, 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000180 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 54032, 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 344016 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-25T15:14:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 255, 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.038215680, 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000149865 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000006031, 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000003015 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-25T15:14:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 152, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 152, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 152, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.122051218, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000802968 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-25T15:14:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 19, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 25, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 2, 2024-04-25T15:14:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 23, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 15, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 2, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 2, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 193, 2024-04-25T15:14:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 185, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-25T15:14:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-04-25T15:14:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 6, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T15:14:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 5, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T15:14:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 14, 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2, 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 17 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-25T15:14:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-25T15:14:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-25T15:14:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-25T15:14:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 101, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 37, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 37, 2024-04-25T15:14:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 64, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 36, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 3, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 2, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 2, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 1, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 1, 2024-04-25T15:14:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 1, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4260878, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4284314, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 83, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.151187895, 2024-04-25T15:14:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001821540 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 23, 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 23, 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-25T15:14:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 18, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 18, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 25, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 25, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 2, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 17, 2024-04-25T15:14:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 15, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 54352, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 231800 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 174, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 152, 2024-04-25T15:14:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 31, 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003850053, 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000962513 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-25T15:14:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 103, 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.023874436, 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000231790 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001643029, 2024-04-25T15:14:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000410757 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-25T15:14:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-25T15:14:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-25T15:14:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-25T15:14:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-25T15:14:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-25T15:14:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001903340, 2024-04-25T15:14:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000190334 2024-04-25T15:14:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-25T15:14:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6, 2024-04-25T15:14:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.008665782, 2024-04-25T15:14:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001444297 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-25T15:14:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.041059408, 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004105940 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000303192, 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000303192 2024-04-25T15:14:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-25T15:14:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002998185, 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001499092 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.016054273, 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004013568 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001121070, 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001121070 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-25T15:14:54.689 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 7, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.020578450, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002939778 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-25T15:14:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-04-25T15:14:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 5, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 3, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 27, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 13, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 609, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T15:14:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 338947, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 135, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 616, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 7, 2024-04-25T15:14:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8032, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 167, 2024-04-25T15:14:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 93946, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 545, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T15:14:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 671, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 115564, 2024-04-25T15:14:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 34, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 671, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 671, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 20.021502182, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.029838304 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-25T15:14:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 671, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 738 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 671, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 0, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 671, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-25T15:14:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 1, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 22, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 0, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 83, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 26, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-25T15:14:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 506, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 44, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-25T15:14:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-25T15:14:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-25T15:14:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-25T15:14:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 99, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 12, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 48, 2024-04-25T15:14:54.709 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 8 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-25T15:14:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-25T15:14:54.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:54.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2631, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 2214888, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2631, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:14:54.713 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2631, 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2214888, 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-25T15:14:54.714 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 671, 2024-04-25T15:14:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 115564, 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 671, 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 115564, 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:54.717 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 671, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 671, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 671, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 671, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T15:14:54.719 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.720 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55c61507a500": { 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 101, 2024-04-25T15:14:54.721 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 90010, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 101, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 83, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 90010, 2024-04-25T15:14:54.722 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:54.723 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T15:14:55.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:54 smithi083 ceph-mon[32919]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:55.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:54 smithi102 ceph-mon[33005]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:55.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:54 smithi019 ceph-mon[28090]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:14:55.361 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T15:14:55.362 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T15:14:55.362 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T15:14:55.362 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T15:14:55.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T15:14:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:14:56.735 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:56 smithi083 ceph-mon[32919]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:57.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:56 smithi102 ceph-mon[33005]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:57.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:56 smithi019 ceph-mon[28090]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:14:57.334 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T15:14:57.334 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:14:57.334 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 227, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 216, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1591008, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 133024, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.047800721, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014523286, 2024-04-25T15:14:57.335 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.035090985, 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.005456522, 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 215, 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.012170954, 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000056609 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:57.336 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1591008, 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 133024 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:14:57.337 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 167, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 480, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 263456, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.011589545, 2024-04-25T15:14:57.338 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.007601408, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001717533, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 163, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.019145817, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000117459 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 263456 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.340 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.000642630, 2024-04-25T15:14:57.341 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000032702, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000568789, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000664576, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000221525 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.342 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:14:57.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T15:14:57.344 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:14:57.345 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:14:57.346 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T15:14:57.347 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:14:57.348 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:14:57.349 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T15:14:57.350 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-25T15:14:57.351 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-25T15:14:57.351 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:14:57.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 230, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1559037, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 230, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 230, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1559037, 2024-04-25T15:14:57.353 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:14:57.354 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:57.993 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:57.994 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:57 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.023 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T15:14:58.023 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T15:14:58.023 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T15:14:58.023 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T15:14:58.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T15:14:58.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:14:58.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:57 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:57 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:14:58.903 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:14:59.029 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:58 smithi083 ceph-mon[32919]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:59.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:58 smithi019 ceph-mon[28090]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:14:59.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:58 smithi102 ceph-mon[33005]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:15:00.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:14:59 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:15:00.163 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.165 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.165 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.165 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:15:00.165 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:14:59 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:15:00.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:00.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-25T15:15:00.309 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:14:59 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi102 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-25T15:15:00.549 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T15:15:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001071363, 2024-04-25T15:15:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000183035, 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:15:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 7, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 168, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3424, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 263904, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.012461238, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.008108894, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002224489, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 166, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.028954530, 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000174424 2024-04-25T15:15:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 263904 2024-04-25T15:15:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 226, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 215, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1590736, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 132384, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.046033248, 2024-04-25T15:15:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014520980, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.034287051, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.005571072, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 214, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.011874018, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000055486 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1590736, 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 132384 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T15:15:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T15:15:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 229, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1558907, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 229, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 229, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1558907, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:15:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:15:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T15:15:01.384 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-25T15:15:01.384 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-25T15:15:01.384 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-25T15:15:01.384 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T15:15:01.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T15:15:01.636 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:15:01.758 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:01 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:01.759 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:01 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:01.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:01 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:01.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:01 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:02.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:01 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:02.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:01 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:02.744 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.745 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.746 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:15:02.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:02 smithi083 ceph-mon[32919]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:15:03.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:15:03.060 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:02 smithi102 ceph-mon[33005]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:15:03.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:02 smithi019 ceph-mon[28090]: from='mgr.15015 ' entity='mgr.x' 2024-04-25T15:15:03.257 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T15:15:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 231, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 220, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1592096, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 135584, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.047536054, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014354760, 2024-04-25T15:15:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.035685326, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.005430326, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 219, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.011081957, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000050602 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1592096, 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 135584 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-04-25T15:15:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 168, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 185552, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.011957913, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.008244921, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001395752, 2024-04-25T15:15:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 165, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.030104803, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000182453 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:15:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 185552 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.000526617, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000038503, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000446550, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000640427, 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000213475 2024-04-25T15:15:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:15:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148692, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-25T15:15:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T15:15:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 233, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1559530, 2024-04-25T15:15:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 233, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 233, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1559530, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T15:15:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T15:15:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T15:15:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:15:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T15:15:03.277 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T15:15:03.918 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:03 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:15:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:03 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:15:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:03 smithi083 ceph-mon[32919]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:15:03.919 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:03 smithi083 ceph-mon[32919]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:15:03.946 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T15:15:03.946 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T15:15:03.946 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T15:15:03.946 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T15:15:03.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T15:15:03.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:03 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:15:03.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:03 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:15:03.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:03 smithi102 ceph-mon[33005]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:15:03.973 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:03 smithi102 ceph-mon[33005]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:15:04.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:03 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-04-25T15:15:04.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:03 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T15:15:04.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:03 smithi019 ceph-mon[28090]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T15:15:04.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:03 smithi019 ceph-mon[28090]: Unable to set osd_memory_target on smithi083 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-04-25T15:15:04.195 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:15:05.005 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:04 smithi102 ceph-mon[33005]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:05.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:04 smithi083 ceph-mon[32919]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:05.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:04 smithi019 ceph-mon[28090]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:15:05.893 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:05.894 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001098618, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000219635, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:05.895 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.896 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 232, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 219, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1594224, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 134112, 2024-04-25T15:15:05.897 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.046338902, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.014515019, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.034648845, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005638123, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 219, 2024-04-25T15:15:05.898 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010990263, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000050183 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1594224, 2024-04-25T15:15:05.899 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 134112 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 166, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 185200, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-25T15:15:05.900 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.010756449, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.007092610, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001369876, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 163, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.027037609, 2024-04-25T15:15:05.901 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000165874 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 185200 2024-04-25T15:15:05.902 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:05.903 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:05.904 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:05.905 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:05.906 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:05.907 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:05.908 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:05.909 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:05.910 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 230, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1559140, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 230, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-25T15:15:05.911 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 230, 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1559140, 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:05.912 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:05.913 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-25T15:15:06.533 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T15:15:06.533 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T15:15:06.533 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T15:15:06.534 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T15:15:06.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T15:15:06.807 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:06 smithi102 ceph-mon[33005]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:15:06.811 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:15:07.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:06 smithi083 ceph-mon[32919]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:15:07.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:06 smithi019 ceph-mon[28090]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:15:08.363 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4, 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 448, 2024-04-25T15:15:08.364 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001269820, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000030300, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000451441, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000654586, 2024-04-25T15:15:08.365 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000218195 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:15:08.366 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 229, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 218, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1591552, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 134304, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:08.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.048121009, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.014374420, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.037340339, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005768714, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 217, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012211580, 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000056274 2024-04-25T15:15:08.368 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1591552, 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 134304 2024-04-25T15:15:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 3, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 168, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 480, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 263632, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:08.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.011159750, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.007401208, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001483681, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 164, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.029871077, 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000182140 2024-04-25T15:15:08.371 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 263632 2024-04-25T15:15:08.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:08.373 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:08.374 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:08.375 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:08.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:08.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:08.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:08.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:08.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:08.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:08.378 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:08.379 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-25T15:15:08.380 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-25T15:15:08.381 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 232, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1559297, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 232, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 232, 2024-04-25T15:15:08.382 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1559297, 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:08.383 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-25T15:15:09.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:08 smithi083 ceph-mon[32919]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:15:09.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:08 smithi102 ceph-mon[33005]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:15:09.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:08 smithi019 ceph-mon[28090]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:15:09.161 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T15:15:09.162 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T15:15:09.162 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T15:15:09.162 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T15:15:09.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T15:15:09.401 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:15:09.802 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:09 smithi102 ceph-mon[33005]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T15:15:10.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:09 smithi083 ceph-mon[32919]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T15:15:10.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:09 smithi019 ceph-mon[28090]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T15:15:10.700 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:10 smithi102 ceph-mon[33005]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:15:10.952 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001118617, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000196711, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:10.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:10.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 228, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 217, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1591280, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 133664, 2024-04-25T15:15:10.955 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.043097735, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.013591367, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.031799662, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005213928, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 216, 2024-04-25T15:15:10.956 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010978205, 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000050825 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.957 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1591280, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 133664 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 6, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 168, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 3264, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 185648, 2024-04-25T15:15:10.958 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.012062806, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.008278776, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001423815, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 166, 2024-04-25T15:15:10.959 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.015809759, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000095239 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:15:10.960 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 185648 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:10.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:10.962 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:10.963 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:10.964 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:10.965 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:10.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-25T15:15:10.967 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-25T15:15:10.968 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 230, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1559140, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 230, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 230, 2024-04-25T15:15:10.969 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1559140, 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:10.970 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-25T15:15:11.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:10 smithi083 ceph-mon[32919]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:11.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:10 smithi019 ceph-mon[28090]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:11.573 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-04-25T15:15:11.574 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-04-25T15:15:11.574 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-04-25T15:15:11.574 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T15:15:11.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 42f950da-0313-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T15:15:11.813 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:15:12.969 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:12 smithi102 ceph-mon[33005]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:13.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:12 smithi083 ceph-mon[32919]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:13.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:12 smithi019 ceph-mon[28090]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:13.488 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-25T15:15:13.488 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 235, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 224, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1593184, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 138144, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.048339284, 2024-04-25T15:15:13.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.014807937, 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.035490127, 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005421557, 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 223, 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012165218, 2024-04-25T15:15:13.490 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000054552 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1593184, 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 138144 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 3, 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 173, 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 480, 2024-04-25T15:15:13.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 264688, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.012277468, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.008400044, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001488560, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:13.493 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 170, 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.019313320, 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000113607 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:13.494 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 264688 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.495 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 448, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-25T15:15:13.496 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.000668046, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000049051, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000595906, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000746952, 2024-04-25T15:15:13.497 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000248984 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:15:13.498 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-25T15:15:13.499 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:15:13.500 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:15:13.501 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:15:13.502 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:15:13.503 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:15:13.504 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148692, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-25T15:15:13.505 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:15:13.506 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-25T15:15:13.506 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:15:13.507 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-25T15:15:13.508 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-25T15:15:13.509 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 238, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1560077, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 238, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-25T15:15:13.510 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 238, 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1560077, 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-25T15:15:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-25T15:15:14.154 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T15:15:14.154 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T15:15:14.154 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T15:15:14.155 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T15:15:14.168 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T15:15:14.168 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T15:15:14.168 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:15:14.235 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T15:15:14.235 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:15:14.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:15:14.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 -- 172.21.15.19:0/3263399205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60081083f0 msgr2=0x7f6008108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 --2- 172.21.15.19:0/3263399205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60081083f0 0x7f6008108850 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009860 tx=0x7f5ff002a3d0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 -- 172.21.15.19:0/3263399205 shutdown_connections 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 --2- 172.21.15.19:0/3263399205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6008108d90 0x7f6008115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 --2- 172.21.15.19:0/3263399205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60081083f0 0x7f6008108850 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 --2- 172.21.15.19:0/3263399205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f600810e460 0x7f600810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 -- 172.21.15.19:0/3263399205 >> 172.21.15.19:0/3263399205 conn(0x7f6008077ad0 msgr2=0x7f6008079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:14.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 -- 172.21.15.19:0/3263399205 shutdown_connections 2024-04-25T15:15:14.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.516+0000 7f600d600640 1 -- 172.21.15.19:0/3263399205 wait complete. 2024-04-25T15:15:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.517+0000 7f600d600640 1 Processor -- start 2024-04-25T15:15:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 -- start start 2024-04-25T15:15:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6008108d90 0x7f60081a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 0x7f600819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6008118640 con 0x7f60081083f0 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60081184c0 con 0x7f600810e460 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f60081187c0 con 0x7f6008108d90 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f6007e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f6007e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40624/0 (socket says 172.21.15.19:40624) 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f6007e00640 1 -- 172.21.15.19:0/3554848094 learned_addr learned my addr 172.21.15.19:0/3554848094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.518+0000 7f600cc00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 0x7f600819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f5fffe00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6008108d90 0x7f60081a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 -- 172.21.15.19:0/3554848094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6008108d90 msgr2=0x7f60081a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6008108d90 0x7f60081a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 -- 172.21.15.19:0/3554848094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 msgr2=0x7f600819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 0x7f600819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 -- 172.21.15.19:0/3554848094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600819f060 con 0x7f60081083f0 2024-04-25T15:15:14.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f600cc00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 0x7f600819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.519+0000 7f6007e00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e9c0 tx=0x7f5ff800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.520+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800cde0 con 0x7f60081083f0 2024-04-25T15:15:14.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.520+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff80035f0 con 0x7f60081083f0 2024-04-25T15:15:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.520+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600819f350 con 0x7f60081083f0 2024-04-25T15:15:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.520+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8010690 con 0x7f60081083f0 2024-04-25T15:15:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.520+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081ab5f0 con 0x7f60081083f0 2024-04-25T15:15:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.521+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60080690e0 con 0x7f60081083f0 2024-04-25T15:15:14.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.523+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ff8004000 con 0x7f60081083f0 2024-04-25T15:15:14.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.525+0000 7f6005600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 0x7f5fec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.525+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f5ff80d0b60 con 0x7f60081083f0 2024-04-25T15:15:14.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.525+0000 7f5fffe00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 0x7f5fec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.526+0000 7f5fffe00640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 0x7f5fec0a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60081a00d0 tx=0x7f5ff0005c90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:14.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.529+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ff8009510 con 0x7f60081083f0 2024-04-25T15:15:14.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.740+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60081102c0 con 0x7f60081083f0 2024-04-25T15:15:14.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.744+0000 7f6005600640 1 -- 172.21.15.19:0/3554848094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23047 (secure 0 0 0) 0x7f5ff809d1a0 con 0x7f60081083f0 2024-04-25T15:15:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:15:14.747 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":101,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:15:13.582387+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 msgr2=0x7f5fec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 0x7f5fec0a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60081a00d0 tx=0x7f5ff0005c90 comp rx=0 tx=0).stop 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 msgr2=0x7f60081a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e9c0 tx=0x7f5ff800ee80 comp rx=0 tx=0).stop 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 shutdown_connections 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f600810e460 0x7f600819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f5fec0a3d20 0x7f5fec0a61e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6008108d90 0x7f60081a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 --2- 172.21.15.19:0/3554848094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60081083f0 0x7f60081a4730 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 >> 172.21.15.19:0/3554848094 conn(0x7f6008077ad0 msgr2=0x7f60080791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:14.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.748+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 shutdown_connections 2024-04-25T15:15:14.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.749+0000 7f600d600640 1 -- 172.21.15.19:0/3554848094 wait complete. 2024-04-25T15:15:14.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:14 smithi019 ceph-mon[28090]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:14.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:14 smithi019 ceph-mon[28090]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T15:15:14.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:56:44.016863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T15:01:58.020978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T15:01:59.501171+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:15:14.771 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:15:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 -- 172.21.15.19:0/2539682971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90010e5f0 msgr2=0x7fd90010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 --2- 172.21.15.19:0/2539682971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90010e5f0 0x7fd90010e9d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009930 tx=0x7fd8f402a420 comp rx=0 tx=0).stop 2024-04-25T15:15:14.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 -- 172.21.15.19:0/2539682971 shutdown_connections 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 --2- 172.21.15.19:0/2539682971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108f40 0x7fd9001153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 --2- 172.21.15.19:0/2539682971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd900108580 0x7fd900108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 --2- 172.21.15.19:0/2539682971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd90010e5f0 0x7fd90010e9d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.992+0000 7fd907400640 1 -- 172.21.15.19:0/2539682971 >> 172.21.15.19:0/2539682971 conn(0x7fd900076490 msgr2=0x7fd9000788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.993+0000 7fd907400640 1 -- 172.21.15.19:0/2539682971 shutdown_connections 2024-04-25T15:15:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.993+0000 7fd907400640 1 -- 172.21.15.19:0/2539682971 wait complete. 2024-04-25T15:15:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 Processor -- start 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 -- start start 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108580 0x7fd90019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90010e5f0 0x7fd9001a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001187b0 con 0x7fd90010e5f0 2024-04-25T15:15:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd900118630 con 0x7fd900108580 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd907400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd900118930 con 0x7fd900108f40 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.994+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:48642/0 (socket says 172.21.15.19:48642) 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 -- 172.21.15.19:0/2810794677 learned_addr learned my addr 172.21.15.19:0/2810794677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd905400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90010e5f0 0x7fd9001a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd904a00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108580 0x7fd90019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 -- 172.21.15.19:0/2810794677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108580 msgr2=0x7fd90019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108580 0x7fd90019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 -- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90010e5f0 msgr2=0x7fd9001a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90010e5f0 0x7fd9001a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 -- 172.21.15.19:0/2810794677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9001a3d80 con 0x7fd900108f40 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8ffe00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9f0 tx=0x7fd8ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.995+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00cd80 con 0x7fd900108f40 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.996+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8ec0108e0 con 0x7fd900108f40 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.996+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec018980 con 0x7fd900108f40 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.996+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9001a4070 con 0x7fd900108f40 2024-04-25T15:15:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.996+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9001ab900 con 0x7fd900108f40 2024-04-25T15:15:15.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:14.997+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9000691c0 con 0x7fd900108f40 2024-04-25T15:15:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.000+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd8ec010430 con 0x7fd900108f40 2024-04-25T15:15:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.001+0000 7fd8fd600640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 0x7fd8e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.001+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fd8ec0d0ad0 con 0x7fd900108f40 2024-04-25T15:15:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.001+0000 7fd904a00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 0x7fd8e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.002+0000 7fd904a00640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 0x7fd8e80a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4031440 tx=0x7fd8f4005a20 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.005+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8ec09cac0 con 0x7fd900108f40 2024-04-25T15:15:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:14 smithi083 ceph-mon[32919]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:15.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:14 smithi083 ceph-mon[32919]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T15:15:15.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:14 smithi102 ceph-mon[33005]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:15:15.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:14 smithi102 ceph-mon[33005]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T15:15:15.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.217+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd90002d5c0 con 0x7fd900108f40 2024-04-25T15:15:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.218+0000 7fd8fd600640 1 -- 172.21.15.19:0/2810794677 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fd8ec0a0dc0 con 0x7fd900108f40 2024-04-25T15:15:15.220 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T15:15:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 msgr2=0x7fd8e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 0x7fd8e80a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4031440 tx=0x7fd8f4005a20 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 msgr2=0x7fd90019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9f0 tx=0x7fd8ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 shutdown_connections 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd90010e5f0 0x7fd9001a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd8e80a3da0 0x7fd8e80a6260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd900108f40 0x7fd90019f2d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 --2- 172.21.15.19:0/2810794677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd900108580 0x7fd90019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 >> 172.21.15.19:0/2810794677 conn(0x7fd900076490 msgr2=0x7fd900077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:15.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 shutdown_connections 2024-04-25T15:15:15.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.222+0000 7fd907400640 1 -- 172.21.15.19:0/2810794677 wait complete. 2024-04-25T15:15:15.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:15:15.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 -- 172.21.15.19:0/1252248178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108580 msgr2=0x7f13dc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 --2- 172.21.15.19:0/1252248178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108580 0x7f13dc108a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009be0 tx=0x7f13c802a6c0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 -- 172.21.15.19:0/1252248178 shutdown_connections 2024-04-25T15:15:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 --2- 172.21.15.19:0/1252248178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108f40 0x7f13dc1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 --2- 172.21.15.19:0/1252248178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108580 0x7f13dc108a00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 --2- 172.21.15.19:0/1252248178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 -- 172.21.15.19:0/1252248178 >> 172.21.15.19:0/1252248178 conn(0x7f13dc076490 msgr2=0x7f13dc0788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:15.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 -- 172.21.15.19:0/1252248178 shutdown_connections 2024-04-25T15:15:15.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.452+0000 7f13e2600640 1 -- 172.21.15.19:0/1252248178 wait complete. 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.453+0000 7f13e2600640 1 Processor -- start 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.453+0000 7f13e2600640 1 -- start start 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 0x7f13dc1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108f40 0x7f13dc1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc118760 con 0x7f13dc108f40 2024-04-25T15:15:15.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1185e0 con 0x7f13dc108580 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13e2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1188e0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:48662/0 (socket says 172.21.15.19:48662) 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13dbe00640 1 -- 172.21.15.19:0/3734511342 learned_addr learned my addr 172.21.15.19:0/3734511342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 0x7f13dc1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.454+0000 7f13daa00640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108f40 0x7f13dc1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 -- 172.21.15.19:0/3734511342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 msgr2=0x7f13dc1a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 0x7f13dc1a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 -- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108f40 msgr2=0x7f13dc1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108f40 0x7f13dc1a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 -- 172.21.15.19:0/3734511342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc19f220 con 0x7f13dc10e5f0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13db400640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 0x7f13dc1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13dbe00640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13d400e9b0 tx=0x7f13d400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d400cd80 con 0x7f13dc10e5f0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc19f510 con 0x7f13dc10e5f0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d40035f0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4010690 con 0x7f13dc10e5f0 2024-04-25T15:15:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.455+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc1ab7b0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.457+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13dc19ffd0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.459+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13d4004020 con 0x7f13dc10e5f0 2024-04-25T15:15:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.460+0000 7f13d3e00640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 0x7f13c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.461+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f13d40d0ae0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.461+0000 7f13db400640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 0x7f13c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.461+0000 7f13db400640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 0x7f13c00a6260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0096c0 tx=0x7f13cc009260 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.465+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13d409d120 con 0x7f13dc10e5f0 2024-04-25T15:15:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.678+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13dc02d5c0 con 0x7f13dc10e5f0 2024-04-25T15:15:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.679+0000 7f13d3e00640 1 -- 172.21.15.19:0/3734511342 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f13d409ca20 con 0x7f13dc10e5f0 2024-04-25T15:15:15.681 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 msgr2=0x7f13c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 0x7f13c00a6260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0096c0 tx=0x7f13cc009260 comp rx=0 tx=0).stop 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 msgr2=0x7f13dc19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13d400e9b0 tx=0x7f13d400ee70 comp rx=0 tx=0).stop 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 shutdown_connections 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f13c00a3da0 0x7f13c00a6260 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13dc10e5f0 0x7f13dc19eaa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13dc108f40 0x7f13dc1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 --2- 172.21.15.19:0/3734511342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13dc108580 0x7f13dc1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.682+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 >> 172.21.15.19:0/3734511342 conn(0x7f13dc076490 msgr2=0x7f13dc077e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:15.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.683+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 shutdown_connections 2024-04-25T15:15:15.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.683+0000 7f13e2600640 1 -- 172.21.15.19:0/3734511342 wait complete. 2024-04-25T15:15:15.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:15:15.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3554848094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:15.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:15 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2810794677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:15:15.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 -- 172.21.15.19:0/2121932571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 msgr2=0x7f1ba0115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 --2- 172.21.15.19:0/2121932571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba0115080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009ca0 tx=0x7f1b9802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 -- 172.21.15.19:0/2121932571 shutdown_connections 2024-04-25T15:15:15.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 --2- 172.21.15.19:0/2121932571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba0115080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 --2- 172.21.15.19:0/2121932571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba0112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 --2- 172.21.15.19:0/2121932571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 0x7f1ba0076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 -- 172.21.15.19:0/2121932571 >> 172.21.15.19:0/2121932571 conn(0x7f1ba00ffa60 msgr2=0x7f1ba0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:15.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 -- 172.21.15.19:0/2121932571 shutdown_connections 2024-04-25T15:15:15.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.921+0000 7f1b9f200640 1 -- 172.21.15.19:0/2121932571 wait complete. 2024-04-25T15:15:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.922+0000 7f1b9f200640 1 Processor -- start 2024-04-25T15:15:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 -- start start 2024-04-25T15:15:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 0x7f1ba01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba01a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01186b0 con 0x7f1ba0112c90 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0118530 con 0x7f1ba0076da0 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0118830 con 0x7f1ba0076480 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40692/0 (socket says 172.21.15.19:40692) 2024-04-25T15:15:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9e800640 1 -- 172.21.15.19:0/3548739038 learned_addr learned my addr 172.21.15.19:0/3548739038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.923+0000 7f1b9d400640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba01a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9de00640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 0x7f1ba01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 -- 172.21.15.19:0/3548739038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 msgr2=0x7f1ba01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 0x7f1ba01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 -- 172.21.15.19:0/3548739038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 msgr2=0x7f1ba01a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba01a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 -- 172.21.15.19:0/3548739038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019f150 con 0x7f1ba0112c90 2024-04-25T15:15:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9d400640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.924+0000 7f1b9e800640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005e30 tx=0x7f1b980316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.925+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98003b70 con 0x7f1ba0112c90 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.925+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b980318e0 con 0x7f1ba0112c90 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.925+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba019f3e0 con 0x7f1ba0112c90 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.925+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9802f990 con 0x7f1ba0112c90 2024-04-25T15:15:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.925+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba01ab690 con 0x7f1ba0112c90 2024-04-25T15:15:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.926+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00690e0 con 0x7f1ba0112c90 2024-04-25T15:15:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.928+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b9802fb30 con 0x7f1ba0112c90 2024-04-25T15:15:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.930+0000 7f1b96a00640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 0x7f1b840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.930+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f1b980f19d0 con 0x7f1ba0112c90 2024-04-25T15:15:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.930+0000 7f1b9de00640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 0x7f1b840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:15.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.931+0000 7f1b9de00640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 0x7f1b840a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ba01141e0 tx=0x7f1b90005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:15.934+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b980be120 con 0x7f1ba0112c90 2024-04-25T15:15:16.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3554848094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:16.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:15 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2810794677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:15:16.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3554848094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:16.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:15 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2810794677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:15:16.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.146+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ba010de20 con 0x7f1ba0112c90 2024-04-25T15:15:16.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.146+0000 7f1b96a00640 1 -- 172.21.15.19:0/3548739038 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f1b980bda20 con 0x7f1ba0112c90 2024-04-25T15:15:16.148 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T15:15:16.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 msgr2=0x7f1b840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 0x7f1b840a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ba01141e0 tx=0x7f1b90005f90 comp rx=0 tx=0).stop 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 msgr2=0x7f1ba019e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005e30 tx=0x7f1b980316f0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 shutdown_connections 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ba0112c90 0x7f1ba019e9d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba0076da0 0x7f1ba01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f1b840a3d20 0x7f1b840a61e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 --2- 172.21.15.19:0/3548739038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ba0076480 0x7f1ba01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 >> 172.21.15.19:0/3548739038 conn(0x7f1ba00ffa60 msgr2=0x7f1ba0114780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:16.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 shutdown_connections 2024-04-25T15:15:16.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.149+0000 7f1b9f200640 1 -- 172.21.15.19:0/3548739038 wait complete. 2024-04-25T15:15:16.171 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T15:15:16.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 -- 172.21.15.19:0/1721746001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 msgr2=0x7f79d010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 --2- 172.21.15.19:0/1721746001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d010d7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c402a480 comp rx=0 tx=0).stop 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 -- 172.21.15.19:0/1721746001 shutdown_connections 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 --2- 172.21.15.19:0/1721746001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d0108e90 0x7f79d01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 --2- 172.21.15.19:0/1721746001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d01084f0 0x7f79d0108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 --2- 172.21.15.19:0/1721746001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d010d7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 -- 172.21.15.19:0/1721746001 >> 172.21.15.19:0/1721746001 conn(0x7f79d0077d70 msgr2=0x7f79d0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 -- 172.21.15.19:0/1721746001 shutdown_connections 2024-04-25T15:15:16.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.384+0000 7f79d6600640 1 -- 172.21.15.19:0/1721746001 wait complete. 2024-04-25T15:15:16.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.385+0000 7f79d6600640 1 Processor -- start 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.385+0000 7f79d6600640 1 -- start start 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d0108e90 0x7f79d019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d01a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0118700 con 0x7f79d01084f0 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0118580 con 0x7f79d010d3c0 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0118880 con 0x7f79d0108e90 2024-04-25T15:15:16.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40718/0 (socket says 172.21.15.19:40718) 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d5200640 1 -- 172.21.15.19:0/622733910 learned_addr learned my addr 172.21.15.19:0/622733910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.386+0000 7f79d5c00640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d01a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79cfe00640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d0108e90 0x7f79d019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 -- 172.21.15.19:0/622733910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d0108e90 msgr2=0x7f79d019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d0108e90 0x7f79d019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 -- 172.21.15.19:0/622733910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 msgr2=0x7f79d01a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d01a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 -- 172.21.15.19:0/622733910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d01a3d30 con 0x7f79d01084f0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79d5200640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005ab0 tx=0x7f79c4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c403f070 con 0x7f79d01084f0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.387+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c4031d60 con 0x7f79d01084f0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.388+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4035030 con 0x7f79d01084f0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.388+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d01a3fc0 con 0x7f79d01084f0 2024-04-25T15:15:16.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.388+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01ab8b0 con 0x7f79d01084f0 2024-04-25T15:15:16.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.389+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d00690e0 con 0x7f79d01084f0 2024-04-25T15:15:16.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.391+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79c4042070 con 0x7f79d01084f0 2024-04-25T15:15:16.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.392+0000 7f79cd600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:16.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.393+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f79c40f1700 con 0x7f79d01084f0 2024-04-25T15:15:16.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.393+0000 7f79cfe00640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:16.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.394+0000 7f79cfe00640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 0x7f79b80a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005fd0 tx=0x7f79bc005850 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:16.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.395+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79c40bdd40 con 0x7f79d01084f0 2024-04-25T15:15:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.617+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f79d0110900 con 0x7f79d01084f0 2024-04-25T15:15:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.618+0000 7f79cd600640 1 -- 172.21.15.19:0/622733910 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+941 (secure 0 0 0) 0x7f79c40b9030 con 0x7f79d01084f0 2024-04-25T15:15:16.619 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3395740379 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/71619263 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2796401471 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/983233775 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3140116712 2024-04-26T14:53:55.852566+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4125323184 2024-04-26T14:53:55.852566+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2740045356 2024-04-26T14:53:55.852566+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/983233775 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2746348246 2024-04-26T15:04:29.064274+0000 2024-04-25T15:15:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/645310036 2024-04-26T14:53:55.852566+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3512004662 2024-04-26T14:53:20.229663+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1703501063 2024-04-26T14:53:20.229663+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2740045356 2024-04-26T14:53:55.852566+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2453471633 2024-04-26T14:53:20.229663+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2453471633 2024-04-26T14:53:20.229663+0000 2024-04-25T15:15:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2483228355 2024-04-26T14:53:20.229663+0000 2024-04-25T15:15:16.622 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 msgr2=0x7f79b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 0x7f79b80a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005fd0 tx=0x7f79bc005850 comp rx=0 tx=0).stop 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 msgr2=0x7f79d019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005ab0 tx=0x7f79c4003c80 comp rx=0 tx=0).stop 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 shutdown_connections 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d010d3c0 0x7f79d01a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f79b80a3d20 0x7f79b80a61e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d0108e90 0x7f79d019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 --2- 172.21.15.19:0/622733910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d01084f0 0x7f79d019ed40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 >> 172.21.15.19:0/622733910 conn(0x7f79d0077d70 msgr2=0x7f79d00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 shutdown_connections 2024-04-25T15:15:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:16.621+0000 7f79d6600640 1 -- 172.21.15.19:0/622733910 wait complete. 2024-04-25T15:15:16.644 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T15:15:16.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:15:16.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T15:15:16.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:15:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:15:16.750 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T15:15:16.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:15:16.828 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T15:15:16.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:16 smithi019 ceph-mon[28090]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:15:16.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3734511342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:15:16.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:16 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3548739038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:15:16.883 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T15:15:16.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:15:16.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:15:17.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:15:17.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T15:15:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:16 smithi083 ceph-mon[32919]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:15:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3734511342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:15:17.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:16 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3548739038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:15:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:16 smithi102 ceph-mon[33005]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:15:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3734511342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:15:17.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:16 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3548739038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:15:17.154 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T15:15:17.165 INFO:tasks.cephadm:Teardown begin 2024-04-25T15:15:17.165 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T15:15:17.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:15:17.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 -- 172.21.15.19:0/2941338071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8112be0 msgr2=0x7f6ba8114fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 --2- 172.21.15.19:0/2941338071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8112be0 0x7f6ba8114fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009cd0 tx=0x7f6ba002a8f0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 -- 172.21.15.19:0/2941338071 shutdown_connections 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 --2- 172.21.15.19:0/2941338071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8112be0 0x7f6ba8114fd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 --2- 172.21.15.19:0/2941338071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8076da0 0x7f6ba81126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 --2- 172.21.15.19:0/2941338071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 0x7f6ba8076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.352+0000 7f6ba7200640 1 -- 172.21.15.19:0/2941338071 >> 172.21.15.19:0/2941338071 conn(0x7f6ba80ff9f0 msgr2=0x7f6ba8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.353+0000 7f6ba7200640 1 -- 172.21.15.19:0/2941338071 shutdown_connections 2024-04-25T15:15:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.353+0000 7f6ba7200640 1 -- 172.21.15.19:0/2941338071 wait complete. 2024-04-25T15:15:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.353+0000 7f6ba7200640 1 Processor -- start 2024-04-25T15:15:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 -- start start 2024-04-25T15:15:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 0x7f6ba81a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 0x7f6ba819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8118600 con 0x7f6ba8112be0 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8118480 con 0x7f6ba8076480 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8118780 con 0x7f6ba8076da0 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:48716/0 (socket says 172.21.15.19:48716) 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.354+0000 7f6ba5400640 1 -- 172.21.15.19:0/4129970819 learned_addr learned my addr 172.21.15.19:0/4129970819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba6800640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 0x7f6ba819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5e00640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 0x7f6ba81a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 -- 172.21.15.19:0/4129970819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 msgr2=0x7f6ba81a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 0x7f6ba81a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 -- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 msgr2=0x7f6ba819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 0x7f6ba819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 -- 172.21.15.19:0/4129970819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819f120 con 0x7f6ba8076da0 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba6800640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 0x7f6ba819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.355+0000 7f6ba5400640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9f0 tx=0x7f6b9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.356+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000cd80 con 0x7f6ba8076da0 2024-04-25T15:15:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.356+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b900108e0 con 0x7f6ba8076da0 2024-04-25T15:15:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.356+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819f410 con 0x7f6ba8076da0 2024-04-25T15:15:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.356+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90018980 con 0x7f6ba8076da0 2024-04-25T15:15:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.356+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81ab6b0 con 0x7f6ba8076da0 2024-04-25T15:15:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.357+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba80690e0 con 0x7f6ba8076da0 2024-04-25T15:15:17.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.360+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b90010430 con 0x7f6ba8076da0 2024-04-25T15:15:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.361+0000 7f6b9ea00640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 0x7f6b8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.361+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f6b900d0df0 con 0x7f6ba8076da0 2024-04-25T15:15:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.362+0000 7f6ba5e00640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 0x7f6b8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.362+0000 7f6ba5e00640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 0x7f6b8c0a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ba819fae0 tx=0x7f6b98005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:17.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.363+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b9009d430 con 0x7f6ba8076da0 2024-04-25T15:15:17.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.613+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ba802d5c0 con 0x7f6ba8076da0 2024-04-25T15:15:17.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.619+0000 7f6b9ea00640 1 -- 172.21.15.19:0/4129970819 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7f6b9009cd30 con 0x7f6ba8076da0 2024-04-25T15:15:17.621 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:15:17.621 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:15:17.621 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:15:17.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 msgr2=0x7f6b8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 0x7f6b8c0a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ba819fae0 tx=0x7f6b98005f90 comp rx=0 tx=0).stop 2024-04-25T15:15:17.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 msgr2=0x7f6ba81a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9f0 tx=0x7f6b9000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 shutdown_connections 2024-04-25T15:15:17.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ba8112be0 0x7f6ba819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f6b8c0a3d20 0x7f6b8c0a61e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ba8076da0 0x7f6ba81a4d30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 --2- 172.21.15.19:0/4129970819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba8076480 0x7f6ba81a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.622+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 >> 172.21.15.19:0/4129970819 conn(0x7f6ba80ff9f0 msgr2=0x7f6ba8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.623+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 shutdown_connections 2024-04-25T15:15:17.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.623+0000 7f6ba7200640 1 -- 172.21.15.19:0/4129970819 wait complete. 2024-04-25T15:15:17.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:17 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/622733910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:15:17.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:15:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 -- 172.21.15.19:0/742125724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e4074c10 msgr2=0x7fd7e4112820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 --2- 172.21.15.19:0/742125724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e4074c10 0x7fd7e4112820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009860 tx=0x7fd7d002a3d0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 -- 172.21.15.19:0/742125724 shutdown_connections 2024-04-25T15:15:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 --2- 172.21.15.19:0/742125724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e4115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 --2- 172.21.15.19:0/742125724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e4074c10 0x7fd7e4112820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 --2- 172.21.15.19:0/742125724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e40742f0 0x7fd7e40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 -- 172.21.15.19:0/742125724 >> 172.21.15.19:0/742125724 conn(0x7fd7e40fd860 msgr2=0x7fd7e40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 -- 172.21.15.19:0/742125724 shutdown_connections 2024-04-25T15:15:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.855+0000 7fd7ea800640 1 -- 172.21.15.19:0/742125724 wait complete. 2024-04-25T15:15:17.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.856+0000 7fd7ea800640 1 Processor -- start 2024-04-25T15:15:17.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 -- start start 2024-04-25T15:15:17.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e4074c10 0x7fd7e41a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4118640 con 0x7fd7e4074c10 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e41184c0 con 0x7fd7e4112d60 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7ea800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e41187c0 con 0x7fd7e40742f0 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7e8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e4074c10 0x7fd7e41a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.857+0000 7fd7e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.19:48728/0 (socket says 172.21.15.19:48728) 2024-04-25T15:15:17.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 -- 172.21.15.19:0/1080195116 learned_addr learned my addr 172.21.15.19:0/1080195116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9e00640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 -- 172.21.15.19:0/1080195116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 msgr2=0x7fd7e419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 -- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e4074c10 msgr2=0x7fd7e41a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e4074c10 0x7fd7e41a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 -- 172.21.15.19:0/1080195116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e419f1c0 con 0x7fd7e40742f0 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9e00640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.858+0000 7fd7e9400640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c420 tx=0x7fd7d8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.859+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8003970 con 0x7fd7e40742f0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.859+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d8004020 con 0x7fd7e40742f0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.859+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e419f4b0 con 0x7fd7e40742f0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.859+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800b9a0 con 0x7fd7e40742f0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.859+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e41ab750 con 0x7fd7e40742f0 2024-04-25T15:15:17.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.860+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e40690e0 con 0x7fd7e40742f0 2024-04-25T15:15:17.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.862+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7d801d020 con 0x7fd7e40742f0 2024-04-25T15:15:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.863+0000 7fd7e2000640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 0x7fd7cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.864+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fd7d80d0700 con 0x7fd7e40742f0 2024-04-25T15:15:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.864+0000 7fd7e8a00640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 0x7fd7cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.864+0000 7fd7e8a00640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 0x7fd7cc0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7e41a0230 tx=0x7fd7d0005c90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:17.867+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7d809cd40 con 0x7fd7e40742f0 2024-04-25T15:15:18.040 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:17 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/622733910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:15:18.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:17 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/622733910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:15:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.118+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7e402d5c0 con 0x7fd7e40742f0 2024-04-25T15:15:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.119+0000 7fd7e2000640 1 -- 172.21.15.19:0/1080195116 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11804 (secure 0 0 0) 0x7fd7d8098030 con 0x7fd7e40742f0 2024-04-25T15:15:18.121 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:15:18.121 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3801236532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3801236532},{"type":"v1","addr":"172.21.15.19:6837","nonce":3801236532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2415265237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2415265237},{"type":"v1","addr":"172.21.15.19:6839","nonce":2415265237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1625313784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1625313784},{"type":"v1","addr":"172.21.15.102:6833","nonce":1625313784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2427216282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2427216282},{"type":"v1","addr":"172.21.15.83:6833","nonce":2427216282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1454655799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1454655799},{"type":"v1","addr":"172.21.15.19:6835","nonce":1454655799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/334283941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":334283941},{"type":"v1","addr":"172.21.15.83:6835","nonce":334283941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3800814259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3800814259},{"type":"v1","addr":"172.21.15.102:6835","nonce":3800814259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/560569157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":560569157},{"type":"v1","addr":"172.21.15.102:6837","nonce":560569157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3676268156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3676268156},{"type":"v1","addr":"172.21.15.83:6837","nonce":3676268156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/4066270851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":4066270851},{"type":"v1","addr":"172.21.15.102:6839","nonce":4066270851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T15:02:00.518771+0000","modified":"2024-04-25T15:02:04.773178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/2613932433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":2613932433},{"type":"v1","addr":"172.21.15.19:6841","nonce":2613932433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/2949462770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2949462770},{"type":"v1","addr":"172.21.15.83:6839","nonce":2949462770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24590,"qdb_cluster":[24590]},"id":1}]} 2024-04-25T15:15:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T15:15:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 msgr2=0x7fd7cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 0x7fd7cc0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7e41a0230 tx=0x7fd7d0005c90 comp rx=0 tx=0).stop 2024-04-25T15:15:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 msgr2=0x7fd7e41a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800c420 tx=0x7fd7d8009f90 comp rx=0 tx=0).stop 2024-04-25T15:15:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 shutdown_connections 2024-04-25T15:15:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e4112d60 0x7fd7e419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7e4074c10 0x7fd7e41a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fd7cc0a3d20 0x7fd7cc0a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 --2- 172.21.15.19:0/1080195116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e40742f0 0x7fd7e41a4890 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 >> 172.21.15.19:0/1080195116 conn(0x7fd7e40fd860 msgr2=0x7fd7e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.122+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 shutdown_connections 2024-04-25T15:15:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.123+0000 7fd7ea800640 1 -- 172.21.15.19:0/1080195116 wait complete. 2024-04-25T15:15:18.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:15:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 -- 172.21.15.19:0/3745697520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d48069350 msgr2=0x7f3d48069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 --2- 172.21.15.19:0/3745697520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d48069350 0x7f3d48069730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009be0 tx=0x7f3d4002a6c0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 -- 172.21.15.19:0/3745697520 shutdown_connections 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 --2- 172.21.15.19:0/3745697520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d481070f0 0x7f3d4810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 --2- 172.21.15.19:0/3745697520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d48069d00 0x7f3d481069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 --2- 172.21.15.19:0/3745697520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d48069350 0x7f3d48069730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 -- 172.21.15.19:0/3745697520 >> 172.21.15.19:0/3745697520 conn(0x7f3d48076790 msgr2=0x7f3d48076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 -- 172.21.15.19:0/3745697520 shutdown_connections 2024-04-25T15:15:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.365+0000 7f3d4f200640 1 -- 172.21.15.19:0/3745697520 wait complete. 2024-04-25T15:15:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.366+0000 7f3d4f200640 1 Processor -- start 2024-04-25T15:15:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.366+0000 7f3d4f200640 1 -- start start 2024-04-25T15:15:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 0x7f3d481a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d4819a600 0x7f3d4819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112000 con 0x7f3d481070f0 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48111e80 con 0x7f3d48069350 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112180 con 0x7f3d4819a600 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 0x7f3d481a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40788/0 (socket says 172.21.15.19:40788) 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4d400640 1 -- 172.21.15.19:0/2481416634 learned_addr learned my addr 172.21.15.19:0/2481416634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.367+0000 7f3d4e800640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d4819a600 0x7f3d4819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 -- 172.21.15.19:0/2481416634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d4819a600 msgr2=0x7f3d4819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d4819a600 0x7f3d4819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 -- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 msgr2=0x7f3d481a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 -- 172.21.15.19:0/2481416634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4819b120 con 0x7f3d48069350 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4d400640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4de00640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 0x7f3d481a0400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d4002abe0 tx=0x7f3d40003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40041020 con 0x7f3d48069350 2024-04-25T15:15:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d400318f0 con 0x7f3d48069350 2024-04-25T15:15:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40036040 con 0x7f3d48069350 2024-04-25T15:15:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4819b3b0 con 0x7f3d48069350 2024-04-25T15:15:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.368+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a7550 con 0x7f3d48069350 2024-04-25T15:15:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.369+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d48068c90 con 0x7f3d48069350 2024-04-25T15:15:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.372+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d40044020 con 0x7f3d48069350 2024-04-25T15:15:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.373+0000 7f3d3ea00640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 0x7f3d340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.374+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f3d400f1780 con 0x7f3d48069350 2024-04-25T15:15:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.374+0000 7f3d4d400640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 0x7f3d340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.375+0000 7f3d4d400640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 0x7f3d340a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3d481084b0 tx=0x7f3d38005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.375+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d4004b020 con 0x7f3d48069350 2024-04-25T15:15:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.584+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d48069dd0 con 0x7f3d48069350 2024-04-25T15:15:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.586+0000 7f3d3ea00640 1 -- 172.21.15.19:0/2481416634 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23047 (secure 0 0 0) 0x7f3d400bddc0 con 0x7f3d48069350 2024-04-25T15:15:18.589 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T15:15:18.589 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":101,"fsid":"42f950da-0313-11ef-bc93-c7b262605968","created":"2024-04-25T14:52:38.625810+0000","modified":"2024-04-25T15:15:13.582387+0000","last_up_change":"2024-04-25T15:01:02.483320+0000","last_in_change":"2024-04-25T15:00:41.583709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:56:44.016863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T15:01:58.020978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T15:01:59.501171+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8649bfaf-7456-4969-a41c-c8752a4bf853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6803","nonce":1027909993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6805","nonce":1027909993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6809","nonce":1027909993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":1027909993},{"type":"v1","addr":"172.21.15.19:6807","nonce":1027909993}]},"public_addr":"172.21.15.19:6803/1027909993","cluster_addr":"172.21.15.19:6805/1027909993","heartbeat_back_addr":"172.21.15.19:6809/1027909993","heartbeat_front_addr":"172.21.15.19:6807/1027909993","state":["exists","up"]},{"osd":1,"uuid":"fb5271f6-eea9-4b6f-8b71-6c3f259b237e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6801","nonce":3419816052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6803","nonce":3419816052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6807","nonce":3419816052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3419816052},{"type":"v1","addr":"172.21.15.83:6805","nonce":3419816052}]},"public_addr":"172.21.15.83:6801/3419816052","cluster_addr":"172.21.15.83:6803/3419816052","heartbeat_back_addr":"172.21.15.83:6807/3419816052","heartbeat_front_addr":"172.21.15.83:6805/3419816052","state":["exists","up"]},{"osd":2,"uuid":"3e2326e2-31a4-4563-9b95-e5da5099e1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6801","nonce":379851586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6803","nonce":379851586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6807","nonce":379851586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":379851586},{"type":"v1","addr":"172.21.15.102:6805","nonce":379851586}]},"public_addr":"172.21.15.102:6801/379851586","cluster_addr":"172.21.15.102:6803/379851586","heartbeat_back_addr":"172.21.15.102:6807/379851586","heartbeat_front_addr":"172.21.15.102:6805/379851586","state":["exists","up"]},{"osd":3,"uuid":"05431369-edfd-411e-9a63-b6b60b86ae1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6811","nonce":3094191876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6813","nonce":3094191876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6817","nonce":3094191876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3094191876},{"type":"v1","addr":"172.21.15.19:6815","nonce":3094191876}]},"public_addr":"172.21.15.19:6811/3094191876","cluster_addr":"172.21.15.19:6813/3094191876","heartbeat_back_addr":"172.21.15.19:6817/3094191876","heartbeat_front_addr":"172.21.15.19:6815/3094191876","state":["exists","up"]},{"osd":4,"uuid":"45e8783f-267c-42c5-834c-905d5e715a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6809","nonce":683494368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6811","nonce":683494368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6815","nonce":683494368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":683494368},{"type":"v1","addr":"172.21.15.83:6813","nonce":683494368}]},"public_addr":"172.21.15.83:6809/683494368","cluster_addr":"172.21.15.83:6811/683494368","heartbeat_back_addr":"172.21.15.83:6815/683494368","heartbeat_front_addr":"172.21.15.83:6813/683494368","state":["exists","up"]},{"osd":5,"uuid":"23ba29b0-bb4a-458a-b988-de82bd6f7629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6809","nonce":4087768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6811","nonce":4087768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6815","nonce":4087768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":4087768},{"type":"v1","addr":"172.21.15.102:6813","nonce":4087768}]},"public_addr":"172.21.15.102:6809/4087768","cluster_addr":"172.21.15.102:6811/4087768","heartbeat_back_addr":"172.21.15.102:6815/4087768","heartbeat_front_addr":"172.21.15.102:6813/4087768","state":["exists","up"]},{"osd":6,"uuid":"e7e6a2a9-70b8-48a9-8dff-2b6f2d164734","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6819","nonce":3457660451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6821","nonce":3457660451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6825","nonce":3457660451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3457660451},{"type":"v1","addr":"172.21.15.19:6823","nonce":3457660451}]},"public_addr":"172.21.15.19:6819/3457660451","cluster_addr":"172.21.15.19:6821/3457660451","heartbeat_back_addr":"172.21.15.19:6825/3457660451","heartbeat_front_addr":"172.21.15.19:6823/3457660451","state":["exists","up"]},{"osd":7,"uuid":"2b6e2fed-875b-495d-aeae-bd3d93a14fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6817","nonce":1101145206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6819","nonce":1101145206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6823","nonce":1101145206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1101145206},{"type":"v1","addr":"172.21.15.83:6821","nonce":1101145206}]},"public_addr":"172.21.15.83:6817/1101145206","cluster_addr":"172.21.15.83:6819/1101145206","heartbeat_back_addr":"172.21.15.83:6823/1101145206","heartbeat_front_addr":"172.21.15.83:6821/1101145206","state":["exists","up"]},{"osd":8,"uuid":"3b498ef1-2388-4402-9170-5c4a8e4cc29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6817","nonce":3832909969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6819","nonce":3832909969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6823","nonce":3832909969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3832909969},{"type":"v1","addr":"172.21.15.102:6821","nonce":3832909969}]},"public_addr":"172.21.15.102:6817/3832909969","cluster_addr":"172.21.15.102:6819/3832909969","heartbeat_back_addr":"172.21.15.102:6823/3832909969","heartbeat_front_addr":"172.21.15.102:6821/3832909969","state":["exists","up"]},{"osd":9,"uuid":"7772cbd2-d2b5-4c1b-beaf-93651cd80d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6827","nonce":2037522312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6829","nonce":2037522312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6833","nonce":2037522312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2037522312},{"type":"v1","addr":"172.21.15.19:6831","nonce":2037522312}]},"public_addr":"172.21.15.19:6827/2037522312","cluster_addr":"172.21.15.19:6829/2037522312","heartbeat_back_addr":"172.21.15.19:6833/2037522312","heartbeat_front_addr":"172.21.15.19:6831/2037522312","state":["exists","up"]},{"osd":10,"uuid":"924537e9-24c0-45be-8ae2-f5623c73902b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6825","nonce":2320025871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6827","nonce":2320025871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6831","nonce":2320025871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2320025871},{"type":"v1","addr":"172.21.15.83:6829","nonce":2320025871}]},"public_addr":"172.21.15.83:6825/2320025871","cluster_addr":"172.21.15.83:6827/2320025871","heartbeat_back_addr":"172.21.15.83:6831/2320025871","heartbeat_front_addr":"172.21.15.83:6829/2320025871","state":["exists","up"]},{"osd":11,"uuid":"6578f249-8ff7-4cd6-b80f-9b7b873d0bc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6825","nonce":2593361750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6827","nonce":2593361750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6831","nonce":2593361750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2593361750},{"type":"v1","addr":"172.21.15.102:6829","nonce":2593361750}]},"public_addr":"172.21.15.102:6825/2593361750","cluster_addr":"172.21.15.102:6827/2593361750","heartbeat_back_addr":"172.21.15.102:6831/2593361750","heartbeat_front_addr":"172.21.15.102:6829/2593361750","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:51.057376+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:16.958985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:41.202967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:15.287234+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:41.778673+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:05.972759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:40.927059+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:07.668968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:59:31.903043+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:07.028964+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:34.323986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:00:59.622155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3395740379":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/71619263":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2796401471":"2024-04-26T15:04:29.064274+0000","172.21.15.19:6800/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/3140116712":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/4125323184":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6801/983233775":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/2746348246":"2024-04-26T15:04:29.064274+0000","172.21.15.19:0/645310036":"2024-04-26T14:53:55.852566+0000","172.21.15.19:0/3512004662":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/1703501063":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6800/2740045356":"2024-04-26T14:53:55.852566+0000","172.21.15.19:6800/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:6801/2453471633":"2024-04-26T14:53:20.229663+0000","172.21.15.19:0/2483228355":"2024-04-26T14:53:20.229663+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:15:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 msgr2=0x7f3d340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 0x7f3d340a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3d481084b0 tx=0x7f3d38005f90 comp rx=0 tx=0).stop 2024-04-25T15:15:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 msgr2=0x7f3d481a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 0x7f3d481a0400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d4002abe0 tx=0x7f3d40003ba0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 shutdown_connections 2024-04-25T15:15:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f3d340a3d20 0x7f3d340a61e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d4819a600 0x7f3d4819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d481070f0 0x7f3d481a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 --2- 172.21.15.19:0/2481416634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d48069350 0x7f3d481a0400 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 >> 172.21.15.19:0/2481416634 conn(0x7f3d48076790 msgr2=0x7f3d4810d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 shutdown_connections 2024-04-25T15:15:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.589+0000 7f3d4f200640 1 -- 172.21.15.19:0/2481416634 wait complete. 2024-04-25T15:15:18.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T15:15:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 -- 172.21.15.19:0/316006212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc24108e90 msgr2=0x7fbc241152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 --2- 172.21.15.19:0/316006212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc24108e90 0x7fbc241152b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009be0 tx=0x7fbc1c02a750 comp rx=0 tx=0).stop 2024-04-25T15:15:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 -- 172.21.15.19:0/316006212 shutdown_connections 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 --2- 172.21.15.19:0/316006212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc24108e90 0x7fbc241152b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 --2- 172.21.15.19:0/316006212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc24108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 --2- 172.21.15.19:0/316006212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2410d3c0 0x7fbc2410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 -- 172.21.15.19:0/316006212 >> 172.21.15.19:0/316006212 conn(0x7fbc24077d70 msgr2=0x7fbc24106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 -- 172.21.15.19:0/316006212 shutdown_connections 2024-04-25T15:15:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.825+0000 7fbc29800640 1 -- 172.21.15.19:0/316006212 wait complete. 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.826+0000 7fbc29800640 1 Processor -- start 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.826+0000 7fbc29800640 1 -- start start 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc24108e90 0x7fbc241a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc2410d3c0 0x7fbc2419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24118690 con 0x7fbc241084f0 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24118510 con 0x7fbc24108e90 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc29800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24118810 con 0x7fbc2410d3c0 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40808/0 (socket says 172.21.15.19:40808) 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc23e00640 1 -- 172.21.15.19:0/119008104 learned_addr learned my addr 172.21.15.19:0/119008104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc23400640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc24108e90 0x7fbc241a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.827+0000 7fbc28e00640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc2410d3c0 0x7fbc2419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 -- 172.21.15.19:0/119008104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc2410d3c0 msgr2=0x7fbc2419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc2410d3c0 0x7fbc2419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 -- 172.21.15.19:0/119008104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc24108e90 msgr2=0x7fbc241a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc24108e90 0x7fbc241a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 -- 172.21.15.19:0/119008104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2419f190 con 0x7fbc241084f0 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc23e00640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9f0 tx=0x7fbc1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400ccc0 con 0x7fbc241084f0 2024-04-25T15:15:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc140108e0 con 0x7fbc241084f0 2024-04-25T15:15:18.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2419f480 con 0x7fbc241084f0 2024-04-25T15:15:18.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.828+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14018880 con 0x7fbc241084f0 2024-04-25T15:15:18.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.829+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc241ab6d0 con 0x7fbc241084f0 2024-04-25T15:15:18.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.830+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc240690e0 con 0x7fbc241084f0 2024-04-25T15:15:18.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.832+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc14010430 con 0x7fbc241084f0 2024-04-25T15:15:18.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.833+0000 7fbc20c00640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 0x7fbc080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:18.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.834+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fbc140d0b70 con 0x7fbc241084f0 2024-04-25T15:15:18.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.834+0000 7fbc23400640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 0x7fbc080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:18.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.834+0000 7fbc23400640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 0x7fbc080a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc24107fd0 tx=0x7fbc10009260 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:18.838+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc1409d1b0 con 0x7fbc241084f0 2024-04-25T15:15:18.871 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:18 smithi019 ceph-mon[28090]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 235 B/s wr, 2 op/s 2024-04-25T15:15:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/4129970819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1080195116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:18.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:18 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/2481416634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:18 smithi083 ceph-mon[32919]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 235 B/s wr, 2 op/s 2024-04-25T15:15:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/4129970819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1080195116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:18 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/2481416634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:18 smithi102 ceph-mon[33005]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 235 B/s wr, 2 op/s 2024-04-25T15:15:19.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/4129970819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1080195116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:18 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/2481416634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:15:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.087+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbc24110900 con 0x7fbc241084f0 2024-04-25T15:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.092+0000 7fbc20c00640 1 -- 172.21.15.19:0/119008104 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fbc1409cab0 con 0x7fbc241084f0 2024-04-25T15:15:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T15:15:19.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 msgr2=0x7fbc080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 0x7fbc080a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc24107fd0 tx=0x7fbc10009260 comp rx=0 tx=0).stop 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 msgr2=0x7fbc241a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9f0 tx=0x7fbc1400edf0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 shutdown_connections 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fbc080a3d20 0x7fbc080a61e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc2410d3c0 0x7fbc2419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc24108e90 0x7fbc241a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 --2- 172.21.15.19:0/119008104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc241084f0 0x7fbc241a4810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.097+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 >> 172.21.15.19:0/119008104 conn(0x7fbc24077d70 msgr2=0x7fbc240783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.098+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 shutdown_connections 2024-04-25T15:15:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.098+0000 7fbc29800640 1 -- 172.21.15.19:0/119008104 wait complete. 2024-04-25T15:15:19.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T15:15:19.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.324+0000 7ff4e5a00640 1 -- 172.21.15.19:0/1698618239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e0074380 msgr2=0x7ff4e0074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.324+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/1698618239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e0074380 0x7ff4e0074760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009ca0 tx=0x7ff4d802a950 comp rx=0 tx=0).stop 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 -- 172.21.15.19:0/1698618239 shutdown_connections 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/1698618239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e010e630 0x7ff4e0110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/1698618239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 0x7ff4e010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/1698618239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e0074380 0x7ff4e0074760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 -- 172.21.15.19:0/1698618239 >> 172.21.15.19:0/1698618239 conn(0x7ff4e00fd920 msgr2=0x7ff4e00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:19.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 -- 172.21.15.19:0/1698618239 shutdown_connections 2024-04-25T15:15:19.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.325+0000 7ff4e5a00640 1 -- 172.21.15.19:0/1698618239 wait complete. 2024-04-25T15:15:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 Processor -- start 2024-04-25T15:15:19.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 -- start start 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 0x7ff4e01a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e010e630 0x7ff4e019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01142b0 con 0x7ff4e0074ca0 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0114130 con 0x7ff4e0074380 2024-04-25T15:15:19.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0114430 con 0x7ff4e010e630 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4df400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 0x7ff4e01a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:59624/0 (socket says 172.21.15.19:59624) 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4e5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e010e630 0x7ff4e019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.326+0000 7ff4dfe00640 1 -- 172.21.15.19:0/3123077764 learned_addr learned my addr 172.21.15.19:0/3123077764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:19.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 -- 172.21.15.19:0/3123077764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e010e630 msgr2=0x7ff4e019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e010e630 0x7ff4e019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 -- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 msgr2=0x7ff4e01a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 0x7ff4e01a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 -- 172.21.15.19:0/3123077764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e019af10 con 0x7ff4e0074380 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.327+0000 7ff4dfe00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8005e30 tx=0x7ff4d8003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.328+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8034040 con 0x7ff4e0074380 2024-04-25T15:15:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.328+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d8030850 con 0x7ff4e0074380 2024-04-25T15:15:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.328+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e019b1a0 con 0x7ff4e0074380 2024-04-25T15:15:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.328+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80046e0 con 0x7ff4e0074380 2024-04-25T15:15:19.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.328+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01a7510 con 0x7ff4e0074380 2024-04-25T15:15:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.329+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e0075a30 con 0x7ff4e0074380 2024-04-25T15:15:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.331+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4d8037020 con 0x7ff4e0074380 2024-04-25T15:15:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.332+0000 7ff4dcc00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.333+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8555+0+0 (secure 0 0 0) 0x7ff4d80f1490 con 0x7ff4e0074380 2024-04-25T15:15:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.333+0000 7ff4df400640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.333+0000 7ff4df400640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 0x7ff4c40a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4e019bea0 tx=0x7ff4c8005f90 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.335+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4d80bdc50 con 0x7ff4e0074380 2024-04-25T15:15:19.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a[28086]: 2024-04-25T15:15:19.089+0000 7fe77ce00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:15:19.372 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: ignoring --setuser ceph since I am not root 2024-04-25T15:15:19.373 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: ignoring --setgroup ceph since I am not root 2024-04-25T15:15:19.373 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mds[69467]: -- 172.21.15.19:0/465044525 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3755dda40 con 0x55c3755b0c00 2024-04-25T15:15:19.373 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: starting mds.a at 2024-04-25T15:15:19.541 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: ignoring --setuser ceph since I am not root 2024-04-25T15:15:19.541 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: ignoring --setgroup ceph since I am not root 2024-04-25T15:15:19.541 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mds[63161]: -- 172.21.15.83:0/1827401577 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b585a3a40 con 0x558b58576c00 2024-04-25T15:15:19.542 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: starting mds.b at 2024-04-25T15:15:19.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:19.583+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff4e019b500 con 0x7ff4e0074380 2024-04-25T15:15:19.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:15:19.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/3123077764' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:19.815 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:19 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:15:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:15:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:15:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:15:20.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T15:15:20.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:15:20.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/3123077764' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:19 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:15:20.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/119008104' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/3123077764' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:19 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.105+0000 7ff4dcc00640 1 -- 172.21.15.19:0/3123077764 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7ff4d80bd550 con 0x7ff4e0074380 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 msgr2=0x7ff4c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 0x7ff4c40a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4e019bea0 tx=0x7ff4c8005f90 comp rx=0 tx=0).stop 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 msgr2=0x7ff4e01a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8005e30 tx=0x7ff4d8003d60 comp rx=0 tx=0).stop 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 shutdown_connections 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4e010e630 0x7ff4e019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e0074ca0 0x7ff4e01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 --2- 172.21.15.19:0/3123077764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0074380 0x7ff4e01a0590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 >> 172.21.15.19:0/3123077764 conn(0x7ff4e00fd920 msgr2=0x7ff4e00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.109+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 shutdown_connections 2024-04-25T15:15:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.110+0000 7ff4e5a00640 1 -- 172.21.15.19:0/3123077764 wait complete. 2024-04-25T15:15:20.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T15:15:20.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 -- 172.21.15.19:0/3363491503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac074ca0 msgr2=0x7fc2ac10e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:20.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 --2- 172.21.15.19:0/3363491503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac10e0f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc298009900 tx=0x7fc29802a480 comp rx=0 tx=0).stop 2024-04-25T15:15:20.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 -- 172.21.15.19:0/3363491503 shutdown_connections 2024-04-25T15:15:20.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 --2- 172.21.15.19:0/3363491503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac10e630 0x7fc2ac110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 --2- 172.21.15.19:0/3363491503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac10e0f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 --2- 172.21.15.19:0/3363491503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074380 0x7fc2ac074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 -- 172.21.15.19:0/3363491503 >> 172.21.15.19:0/3363491503 conn(0x7fc2ac0fd920 msgr2=0x7fc2ac0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:20.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 -- 172.21.15.19:0/3363491503 shutdown_connections 2024-04-25T15:15:20.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.322+0000 7fc2b2200640 1 -- 172.21.15.19:0/3363491503 wait complete. 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.323+0000 7fc2b2200640 1 Processor -- start 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 -- start start 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 0x7fc2ac19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac114250 con 0x7fc2ac074380 2024-04-25T15:15:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1140d0 con 0x7fc2ac10e630 2024-04-25T15:15:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b2200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1143d0 con 0x7fc2ac074ca0 2024-04-25T15:15:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 0x7fc2ac19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40864/0 (socket says 172.21.15.19:40864) 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.324+0000 7fc2b0e00640 1 -- 172.21.15.19:0/1149621689 learned_addr learned my addr 172.21.15.19:0/1149621689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2abe00640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 -- 172.21.15.19:0/1149621689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 msgr2=0x7fc2ac1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 -- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 msgr2=0x7fc2ac1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 -- 172.21.15.19:0/1149621689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac19aef0 con 0x7fc2ac10e630 2024-04-25T15:15:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2abe00640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b0e00640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2b1800640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 0x7fc2ac19a630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.325+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400ce20 con 0x7fc2ac10e630 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.326+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a4005020 con 0x7fc2ac10e630 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.326+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac19b1e0 con 0x7fc2ac10e630 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.326+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4010690 con 0x7fc2ac10e630 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.326+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a74f0 con 0x7fc2ac10e630 2024-04-25T15:15:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.327+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac075a30 con 0x7fc2ac10e630 2024-04-25T15:15:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.329+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2a40054d0 con 0x7fc2ac10e630 2024-04-25T15:15:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.331+0000 7fc2a9600640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.331+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8555+0+0 (secure 0 0 0) 0x7fc2a40d0f70 con 0x7fc2ac10e630 2024-04-25T15:15:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.331+0000 7fc2b0e00640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.332+0000 7fc2b0e00640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 0x7fc2940a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005fd0 tx=0x7fc2a000b480 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.335+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2a409d590 con 0x7fc2ac10e630 2024-04-25T15:15:20.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:20.538+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc2ac068c90 con 0x7fc2ac10e630 2024-04-25T15:15:20.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T15:15:20.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: Cluster is now healthy 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.83:6838/2222037818,v1:172.21.15.83:6839/2222037818] up:boot 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: mds.? [v2:172.21.15.19:6840/4008054696,v1:172.21.15.19:6841/4008054696] up:boot 2024-04-25T15:15:20.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: fsmap 12 up:standby 2024-04-25T15:15:20.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1149621689' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:20.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:20 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T15:15:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:15:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:15:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: Cluster is now healthy 2024-04-25T15:15:21.041 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.83:6838/2222037818,v1:172.21.15.83:6839/2222037818] up:boot 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: mds.? [v2:172.21.15.19:6840/4008054696,v1:172.21.15.19:6841/4008054696] up:boot 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: fsmap 12 up:standby 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1149621689' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:21.042 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:20 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:21.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: Cluster is now healthy 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: from='mgr.15015 172.21.15.19:0/4093516147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:15:21.058 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.83:6838/2222037818,v1:172.21.15.83:6839/2222037818] up:boot 2024-04-25T15:15:21.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: mds.? [v2:172.21.15.19:6840/4008054696,v1:172.21.15.19:6841/4008054696] up:boot 2024-04-25T15:15:21.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: fsmap 12 up:standby 2024-04-25T15:15:21.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1149621689' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:21.059 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:20 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:21.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.113+0000 7fc2a9600640 1 -- 172.21.15.19:0/1149621689 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7fc2ac068c90 con 0x7fc2ac10e630 2024-04-25T15:15:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T15:15:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 msgr2=0x7fc2940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 0x7fc2940a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005fd0 tx=0x7fc2a000b480 comp rx=0 tx=0).stop 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 msgr2=0x7fc2ac19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 0x7fc2ac19a630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a400ef10 comp rx=0 tx=0).stop 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 shutdown_connections 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2ac10e630 0x7fc2ac19a630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7fc2940a3d20 0x7fc2940a61e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2ac074ca0 0x7fc2ac1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 --2- 172.21.15.19:0/1149621689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac074380 0x7fc2ac1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.118+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 >> 172.21.15.19:0/1149621689 conn(0x7fc2ac0fd920 msgr2=0x7fc2ac0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.119+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 shutdown_connections 2024-04-25T15:15:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.119+0000 7fc2b2200640 1 -- 172.21.15.19:0/1149621689 wait complete. 2024-04-25T15:15:21.143 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T15:15:21.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.359+0000 7f8de1c00640 1 -- 172.21.15.19:0/1545235940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc1084f0 msgr2=0x7f8ddc108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.359+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1545235940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc108950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009900 tx=0x7f8dc402a480 comp rx=0 tx=0).stop 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 -- 172.21.15.19:0/1545235940 shutdown_connections 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1545235940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc108e90 0x7f8ddc1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1545235940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc108950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1545235940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 0x7f8ddc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 -- 172.21.15.19:0/1545235940 >> 172.21.15.19:0/1545235940 conn(0x7f8ddc077d70 msgr2=0x7f8ddc106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 -- 172.21.15.19:0/1545235940 shutdown_connections 2024-04-25T15:15:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.360+0000 7f8de1c00640 1 -- 172.21.15.19:0/1545235940 wait complete. 2024-04-25T15:15:21.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 Processor -- start 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 -- start start 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc108e90 0x7f8ddc1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 0x7f8ddc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118700 con 0x7f8ddc10d3c0 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118580 con 0x7f8ddc1084f0 2024-04-25T15:15:21.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8de1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118880 con 0x7f8ddc108e90 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.361+0000 7f8ddbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc108e90 0x7f8ddc1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.19:59660/0 (socket says 172.21.15.19:59660) 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 -- 172.21.15.19:0/1621332581 learned_addr learned my addr 172.21.15.19:0/1621332581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8de1200640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 0x7f8ddc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 -- 172.21.15.19:0/1621332581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc108e90 msgr2=0x7f8ddc1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc108e90 0x7f8ddc1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 -- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 msgr2=0x7f8ddc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 0x7f8ddc19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 -- 172.21.15.19:0/1621332581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc19f1c0 con 0x7f8ddc1084f0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8ddbe00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00eab0 tx=0x7f8dcc00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.362+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00ce40 con 0x7f8ddc1084f0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.363+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dcc011920 con 0x7f8ddc1084f0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.363+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc19f4b0 con 0x7f8ddc1084f0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.363+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc010a30 con 0x7f8ddc1084f0 2024-04-25T15:15:21.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.363+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc1ab750 con 0x7f8ddc1084f0 2024-04-25T15:15:21.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.364+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc0690e0 con 0x7f8ddc1084f0 2024-04-25T15:15:21.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.366+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8dcc010bd0 con 0x7f8ddc1084f0 2024-04-25T15:15:21.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.367+0000 7f8dd8c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 0x7f8dc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:15:21.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.367+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8001+0+0 (secure 0 0 0) 0x7f8dcc0d0b60 con 0x7f8ddc1084f0 2024-04-25T15:15:21.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.367+0000 7f8ddb400640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 0x7f8dc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:15:21.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.368+0000 7f8ddb400640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 0x7f8dc00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005ab0 tx=0x7f8dc4005a20 comp rx=0 tx=0).ready entity=mgr.15015 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:15:21.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.372+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8dcc016030 con 0x7f8ddc1084f0 2024-04-25T15:15:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:21.582+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ddc19ff70 con 0x7f8ddc1084f0 2024-04-25T15:15:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.125+0000 7f8dd8c00640 1 -- 172.21.15.19:0/1621332581 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f8dcc09d3b0 con 0x7f8ddc1084f0 2024-04-25T15:15:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T15:15:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 msgr2=0x7f8dc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 0x7f8dc00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005ab0 tx=0x7f8dc4005a20 comp rx=0 tx=0).stop 2024-04-25T15:15:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 msgr2=0x7f8ddc1a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:15:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00eab0 tx=0x7f8dcc00ef70 comp rx=0 tx=0).stop 2024-04-25T15:15:22.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 shutdown_connections 2024-04-25T15:15:22.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ddc10d3c0 0x7f8ddc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:22.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.19:6800/974415144,v1:172.21.15.19:6801/974415144] conn(0x7f8dc00a3d20 0x7f8dc00a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:22.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ddc108e90 0x7f8ddc1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:22.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 --2- 172.21.15.19:0/1621332581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ddc1084f0 0x7f8ddc1a4890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:15:22.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.130+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 >> 172.21.15.19:0/1621332581 conn(0x7f8ddc077d70 msgr2=0x7f8ddc0783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:15:22.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.131+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 shutdown_connections 2024-04-25T15:15:22.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T15:15:22.131+0000 7f8de1c00640 1 -- 172.21.15.19:0/1621332581 wait complete. 2024-04-25T15:15:22.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:22.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:22.273 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:22.302 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T15:15:22.302 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:15:22.369 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:15:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:15:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-mon[28090]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T15:15:22.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-mon[28090]: pgmap v341: 65 pgs: 65 active+clean; 577 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:15:22.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-mon[28090]: from='client.? 172.21.15.19:0/1621332581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-mon[28090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.386 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:15:22.401 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T15:15:22.402 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T15:15:22.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a 2024-04-25T15:15:22.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:22 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:15:22.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:22 smithi083 ceph-mon[32919]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T15:15:22.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:22 smithi083 ceph-mon[32919]: pgmap v341: 65 pgs: 65 active+clean; 577 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:15:22.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:22 smithi083 ceph-mon[32919]: from='client.? 172.21.15.19:0/1621332581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:22 smithi083 ceph-mon[32919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:22 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:15:22.557 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:22 smithi102 ceph-mon[33005]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T15:15:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:22 smithi102 ceph-mon[33005]: pgmap v341: 65 pgs: 65 active+clean; 577 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:15:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:22 smithi102 ceph-mon[33005]: from='client.? 172.21.15.19:0/1621332581' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.558 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:22 smithi102 ceph-mon[33005]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:15:22.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 systemd[1]: Stopping Ceph mon.a for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a[28086]: 2024-04-25T15:15:22.929+0000 7fe783c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a[28086]: 2024-04-25T15:15:22.929+0000 7fe783c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T15:15:23.371 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:22 smithi019 podman[85852]: 2024-04-25 15:15:22.971704038 +0000 UTC m=+0.172924439 container died 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:15:23.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:23 smithi019 podman[85852]: 2024-04-25 15:15:23.488285633 +0000 UTC m=+0.689506050 container cleanup 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T15:15:23.622 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:23 smithi019 bash[85852]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a 2024-04-25T15:15:24.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 15:15:23 smithi019 podman[85866]: 2024-04-25 15:15:23.797432679 +0000 UTC m=+0.815611674 container remove 6526b989094c8bc6bc601842d4361b1c871d487e5fbf780aa932bc2b99aff2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T15:15:24.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T15:15:24.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:24.518 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T15:15:24.519 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T15:15:24.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.b 2024-04-25T15:15:24.864 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:24 smithi083 systemd[1]: Stopping Ceph mon.b for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:25.247 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-b[32914]: 2024-04-25T15:15:24.861+0000 7f04a25cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:25.248 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-b[32914]: 2024-04-25T15:15:24.861+0000 7f04a25cf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T15:15:25.248 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:24 smithi083 podman[68491]: 2024-04-25 15:15:24.954701216 +0000 UTC m=+0.241225772 container died 8e289efd1121b260b47571cb6a3ceecf5c5e3587e7939d77cfadefc51415928b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T15:15:25.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:25 smithi083 podman[68491]: 2024-04-25 15:15:25.346271555 +0000 UTC m=+0.632796160 container cleanup 8e289efd1121b260b47571cb6a3ceecf5c5e3587e7939d77cfadefc51415928b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T15:15:25.541 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 25 15:15:25 smithi083 bash[68491]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-b 2024-04-25T15:15:25.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T15:15:25.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:25.760 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T15:15:25.760 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T15:15:25.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.c 2024-04-25T15:15:26.056 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:25 smithi102 systemd[1]: Stopping Ceph mon.c for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:26.307 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c[32978]: 2024-04-25T15:15:26.054+0000 7f63784c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:26.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c[32978]: 2024-04-25T15:15:26.054+0000 7f63784c2700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T15:15:26.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:26 smithi102 podman[70220]: 2024-04-25 15:15:26.218483536 +0000 UTC m=+0.270148624 container died 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T15:15:26.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:26 smithi102 podman[70220]: 2024-04-25 15:15:26.52154115 +0000 UTC m=+0.573206243 container cleanup 6766046cd009c5ca0c09f7a062e82e13addc4f89563bcbd8895ac4cbad7b71b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T15:15:26.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Apr 25 15:15:26 smithi102 bash[70220]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mon-c 2024-04-25T15:15:26.950 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T15:15:26.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:26.995 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T15:15:26.996 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T15:15:26.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x 2024-04-25T15:15:27.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:27 smithi019 systemd[1]: Stopping Ceph mgr.x for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:28.035 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:15:27.625+0000 7f4a43e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:28.035 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x[28326]: 2024-04-25T15:15:27.625+0000 7f4a43e00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T15:15:28.035 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:27 smithi019 podman[85972]: 2024-04-25 15:15:27.719228852 +0000 UTC m=+0.258935070 container died 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:15:28.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:28 smithi019 podman[85972]: 2024-04-25 15:15:28.228553286 +0000 UTC m=+0.768259513 container cleanup 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:15:28.372 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:28 smithi019 bash[85972]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x 2024-04-25T15:15:28.872 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 15:15:28 smithi019 podman[85986]: 2024-04-25 15:15:28.477226692 +0000 UTC m=+0.734208757 container remove 3efbb3d4c44ac3f3565b13f5461eb116ef945445f166edbade7ea6402e6aae5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:15:28.955 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T15:15:29.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:29.066 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T15:15:29.066 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T15:15:29.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y 2024-04-25T15:15:29.380 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:29 smithi083 systemd[1]: Stopping Ceph mgr.y for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:29.659 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:29 smithi083 podman[68610]: 2024-04-25 15:15:29.379220679 +0000 UTC m=+0.116735978 container died db06a94539c73326b4ddb2d9e9cbcdc007b5799c62c59dc2904b145aa0cfb061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T15:15:30.036 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:29 smithi083 podman[68610]: 2024-04-25 15:15:29.792707727 +0000 UTC m=+0.530223024 container cleanup db06a94539c73326b4ddb2d9e9cbcdc007b5799c62c59dc2904b145aa0cfb061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:15:30.037 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:29 smithi083 bash[68610]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y 2024-04-25T15:15:30.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T15:15:30.288 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:30 smithi083 podman[68625]: 2024-04-25 15:15:30.036143712 +0000 UTC m=+0.653199966 container remove db06a94539c73326b4ddb2d9e9cbcdc007b5799c62c59dc2904b145aa0cfb061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T15:15:30.288 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:30 smithi083 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T15:15:30.288 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:30 smithi083 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T15:15:30.289 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:30 smithi083 systemd[1]: Stopped Ceph mgr.y for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:15:30.289 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 25 15:15:30 smithi083 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.411s CPU time. 2024-04-25T15:15:30.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:30.302 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T15:15:30.302 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T15:15:30.302 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z 2024-04-25T15:15:30.631 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:30 smithi102 systemd[1]: Stopping Ceph mgr.z for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:30.903 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:30 smithi102 podman[70341]: 2024-04-25 15:15:30.630602543 +0000 UTC m=+0.135025840 container died 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:15:31.153 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:30 smithi102 podman[70341]: 2024-04-25 15:15:30.968449609 +0000 UTC m=+0.472872915 container cleanup 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T15:15:31.153 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:30 smithi102 bash[70341]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z 2024-04-25T15:15:31.386 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T15:15:31.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:31 smithi102 podman[70355]: 2024-04-25 15:15:31.152927367 +0000 UTC m=+0.514998665 container remove 9c0a4a8f88dcdc7616aa0d000ec8cfa3557ecbd88943cc64f3124da391609612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T15:15:31.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:31 smithi102 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-25T15:15:31.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:31 smithi102 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-25T15:15:31.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:31 smithi102 systemd[1]: Stopped Ceph mgr.z for 42f950da-0313-11ef-bc93-c7b262605968. 2024-04-25T15:15:31.406 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Apr 25 15:15:31 smithi102 systemd[1]: ceph-42f950da-0313-11ef-bc93-c7b262605968@mgr.z.service: Consumed 20.398s CPU time. 2024-04-25T15:15:31.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:31.420 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T15:15:31.420 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T15:15:31.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.0 2024-04-25T15:15:31.871 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:31 smithi019 systemd[1]: Stopping Ceph osd.0 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:32.372 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0[42768]: 2024-04-25T15:15:31.986+0000 7fe61d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:32.372 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0[42768]: 2024-04-25T15:15:31.986+0000 7fe61d400700 -1 osd.0 104 *** Got signal Terminated *** 2024-04-25T15:15:32.372 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0[42768]: 2024-04-25T15:15:31.986+0000 7fe61d400700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:15:37.861 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:37 smithi019 podman[86093]: 2024-04-25 15:15:37.530117896 +0000 UTC m=+5.668012389 container died 716c3036df49cf6b0b075b5882f68af656381c565c81d59ad0d8b9fb9e603ec6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T15:15:38.122 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:38 smithi019 podman[86093]: 2024-04-25 15:15:38.013498407 +0000 UTC m=+6.151392926 container cleanup 716c3036df49cf6b0b075b5882f68af656381c565c81d59ad0d8b9fb9e603ec6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:15:38.122 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:38 smithi019 bash[86093]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0 2024-04-25T15:15:38.622 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:38 smithi019 podman[86111]: 2024-04-25 15:15:38.26959286 +0000 UTC m=+0.726767751 container remove 716c3036df49cf6b0b075b5882f68af656381c565c81d59ad0d8b9fb9e603ec6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:15:39.373 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:39 smithi019 podman[86202]: 2024-04-25 15:15:39.026375095 +0000 UTC m=+0.039851092 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:15:39.374 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:39 smithi019 podman[86202]: 2024-04-25 15:15:39.330365041 +0000 UTC m=+0.343841065 container create e3b0a8169bee3be20232189f9a6f31919a12c84e1f9a0b0bac131d855242e974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T15:15:39.872 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:39 smithi019 podman[86202]: 2024-04-25 15:15:39.628340586 +0000 UTC m=+0.641816617 container init e3b0a8169bee3be20232189f9a6f31919a12c84e1f9a0b0bac131d855242e974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T15:15:39.872 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:39 smithi019 podman[86202]: 2024-04-25 15:15:39.64065043 +0000 UTC m=+0.654126460 container start e3b0a8169bee3be20232189f9a6f31919a12c84e1f9a0b0bac131d855242e974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T15:15:39.872 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:39 smithi019 podman[86202]: 2024-04-25 15:15:39.759900172 +0000 UTC m=+0.773376236 container attach e3b0a8169bee3be20232189f9a6f31919a12c84e1f9a0b0bac131d855242e974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T15:15:40.372 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 15:15:40 smithi019 podman[86202]: 2024-04-25 15:15:40.11744492 +0000 UTC m=+1.130920967 container died e3b0a8169bee3be20232189f9a6f31919a12c84e1f9a0b0bac131d855242e974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T15:15:40.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T15:15:41.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:41.030 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T15:15:41.030 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T15:15:41.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.1 2024-04-25T15:15:41.348 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:41 smithi083 systemd[1]: Stopping Ceph osd.1 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:41.791 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1[37511]: 2024-04-25T15:15:41.346+0000 7f634b663700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:41.791 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1[37511]: 2024-04-25T15:15:41.346+0000 7f634b663700 -1 osd.1 104 *** Got signal Terminated *** 2024-04-25T15:15:41.791 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1[37511]: 2024-04-25T15:15:41.346+0000 7f634b663700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:15:47.016 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:46 smithi083 podman[68738]: 2024-04-25 15:15:46.737423228 +0000 UTC m=+5.514383163 container died c43909c297b1441df2f6227ab6b8c83bf7d40aad952ffa3b54d5f66a4eb4eae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T15:15:47.291 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:47 smithi083 podman[68738]: 2024-04-25 15:15:47.105879412 +0000 UTC m=+5.882839346 container cleanup c43909c297b1441df2f6227ab6b8c83bf7d40aad952ffa3b54d5f66a4eb4eae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-25T15:15:47.291 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:47 smithi083 bash[68738]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1 2024-04-25T15:15:47.777 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:47 smithi083 podman[68756]: 2024-04-25 15:15:47.424680117 +0000 UTC m=+0.685977060 container remove c43909c297b1441df2f6227ab6b8c83bf7d40aad952ffa3b54d5f66a4eb4eae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T15:15:48.041 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:47 smithi083 podman[68848]: 2024-04-25 15:15:47.678874242 +0000 UTC m=+0.017937943 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:15:48.041 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:47 smithi083 podman[68848]: 2024-04-25 15:15:47.805148322 +0000 UTC m=+0.144212054 container create 6f84be29825b024f3583387485ed37313958f61bef3bae18ac85efd5edfaca7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T15:15:48.335 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:48 smithi083 podman[68848]: 2024-04-25 15:15:48.045462727 +0000 UTC m=+0.384526421 container init 6f84be29825b024f3583387485ed37313958f61bef3bae18ac85efd5edfaca7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T15:15:48.335 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:48 smithi083 podman[68848]: 2024-04-25 15:15:48.048682936 +0000 UTC m=+0.387746627 container start 6f84be29825b024f3583387485ed37313958f61bef3bae18ac85efd5edfaca7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T15:15:48.335 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:48 smithi083 podman[68848]: 2024-04-25 15:15:48.128476038 +0000 UTC m=+0.467539743 container attach 6f84be29825b024f3583387485ed37313958f61bef3bae18ac85efd5edfaca7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:15:48.605 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 25 15:15:48 smithi083 podman[68848]: 2024-04-25 15:15:48.334537142 +0000 UTC m=+0.673600827 container died 6f84be29825b024f3583387485ed37313958f61bef3bae18ac85efd5edfaca7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:15:49.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T15:15:49.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:49.041 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T15:15:49.042 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T15:15:49.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.2 2024-04-25T15:15:49.402 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:49 smithi102 systemd[1]: Stopping Ceph osd.2 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:49.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2[38151]: 2024-04-25T15:15:49.430+0000 7f74c0f3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:49.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2[38151]: 2024-04-25T15:15:49.430+0000 7f74c0f3b700 -1 osd.2 104 *** Got signal Terminated *** 2024-04-25T15:15:49.808 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2[38151]: 2024-04-25T15:15:49.430+0000 7f74c0f3b700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:15:55.100 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:54 smithi102 podman[70465]: 2024-04-25 15:15:54.829034558 +0000 UTC m=+5.582017342 container died 8c3a72a436509f12fbb3cbab457a33e133d58627a7a3f7d4d9c04134c7b39684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:15:55.490 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:55 smithi102 podman[70465]: 2024-04-25 15:15:55.205568738 +0000 UTC m=+5.958551563 container cleanup 8c3a72a436509f12fbb3cbab457a33e133d58627a7a3f7d4d9c04134c7b39684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T15:15:55.490 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:55 smithi102 bash[70465]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2 2024-04-25T15:15:55.799 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:55 smithi102 podman[70493]: 2024-04-25 15:15:55.490109671 +0000 UTC m=+0.658105683 container remove 8c3a72a436509f12fbb3cbab457a33e133d58627a7a3f7d4d9c04134c7b39684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T15:15:56.057 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:55 smithi102 podman[70587]: 2024-04-25 15:15:55.799504569 +0000 UTC m=+0.096466558 container create eca5e9eb8c90912641b88128b35de460b4d15f3ffded1709f7bab04e80099b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:15:56.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:55 smithi102 podman[70587]: 2024-04-25 15:15:55.721231395 +0000 UTC m=+0.018193400 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:15:56.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:56 smithi102 podman[70587]: 2024-04-25 15:15:56.026870287 +0000 UTC m=+0.323832288 container init eca5e9eb8c90912641b88128b35de460b4d15f3ffded1709f7bab04e80099b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T15:15:56.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:56 smithi102 podman[70587]: 2024-04-25 15:15:56.029858922 +0000 UTC m=+0.326820913 container start eca5e9eb8c90912641b88128b35de460b4d15f3ffded1709f7bab04e80099b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:15:56.319 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:56 smithi102 podman[70587]: 2024-04-25 15:15:56.093607242 +0000 UTC m=+0.390569247 container attach eca5e9eb8c90912641b88128b35de460b4d15f3ffded1709f7bab04e80099b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T15:15:56.606 INFO:journalctl@ceph.osd.2.smithi102.stdout:Apr 25 15:15:56 smithi102 podman[70587]: 2024-04-25 15:15:56.319006084 +0000 UTC m=+0.615968081 container died eca5e9eb8c90912641b88128b35de460b4d15f3ffded1709f7bab04e80099b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T15:15:56.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T15:15:56.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:56.945 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T15:15:56.945 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T15:15:56.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.3 2024-04-25T15:15:57.372 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:57 smithi019 systemd[1]: Stopping Ceph osd.3 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:15:57.873 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:57 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:15:57.572+0000 7fdad4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:57.873 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:57 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:15:57.572+0000 7fdad4e00700 -1 osd.3 104 *** Got signal Terminated *** 2024-04-25T15:15:57.873 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:57 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:15:57.572+0000 7fdad4e00700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:15:57.997 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:15:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:15:57.620+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:15:58.768 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:15:58.464+0000 7fdad0600700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.072877+0000 front 2024-04-25T15:15:36.073129+0000 (oldest deadline 2024-04-25T15:15:57.772764+0000) 2024-04-25T15:15:59.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:15:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:15:58.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:15:59.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:15:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:15:58.779+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:15:59.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:15:58 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:15:58.765+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:15:59.740 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:15:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:15:59.738+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:15:59.741 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:15:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:15:59.432+0000 7fdad0600700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.072877+0000 front 2024-04-25T15:15:36.073129+0000 (oldest deadline 2024-04-25T15:15:57.772764+0000) 2024-04-25T15:15:59.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:15:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:15:59.388+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:00.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:15:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:15:59.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:00.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:15:59 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:15:59.820+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:00.785 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:16:00.446+0000 7fdad0600700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.072877+0000 front 2024-04-25T15:15:36.073129+0000 (oldest deadline 2024-04-25T15:15:57.772764+0000) 2024-04-25T15:16:00.790 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:00.416+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:01.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:00.590+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:01.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:00.677+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:01.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:00.827+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:01.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:00 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:00.783+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:01.788 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:16:01.475+0000 7fdad0600700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.072877+0000 front 2024-04-25T15:15:36.073129+0000 (oldest deadline 2024-04-25T15:15:57.772764+0000) 2024-04-25T15:16:01.790 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:01.450+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:01.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:01.644+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:01.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:02.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:01.785+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:02.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:01 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:01.800+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:02.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:02 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:02.468+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:02.805 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3[49699]: 2024-04-25T15:16:02.442+0000 7fdad0600700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.072877+0000 front 2024-04-25T15:15:36.073129+0000 (oldest deadline 2024-04-25T15:15:57.772764+0000) 2024-04-25T15:16:02.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:02.665+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:02.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:02.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:03.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:02.802+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:03.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:02 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:02.807+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:03.122 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:03 smithi019 podman[86308]: 2024-04-25 15:16:03.045612259 +0000 UTC m=+5.609359484 container died 7dac251e3c8d38eb19156949c42dc032ef245fc95ddec5f13f8cb0b51e393197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T15:16:03.743 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:03 smithi019 podman[86308]: 2024-04-25 15:16:03.537313593 +0000 UTC m=+6.101060875 container cleanup 7dac251e3c8d38eb19156949c42dc032ef245fc95ddec5f13f8cb0b51e393197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T15:16:03.744 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:03 smithi019 bash[86308]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3 2024-04-25T15:16:03.790 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:03 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:03.420+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:04.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:03.589+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:04.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:03.627+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:04.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:03.768+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:04.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:03 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:03.781+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:04.122 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:03 smithi019 podman[86339]: 2024-04-25 15:16:03.741444811 +0000 UTC m=+0.685706491 container remove 7dac251e3c8d38eb19156949c42dc032ef245fc95ddec5f13f8cb0b51e393197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:16:04.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:04 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:04.419+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:04.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:04 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:04.786+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:04.872 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:04 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:04.797+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:04.873 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:04 smithi019 podman[86428]: 2024-04-25 15:16:04.501149099 +0000 UTC m=+0.040929258 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:04.873 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:04 smithi019 podman[86428]: 2024-04-25 15:16:04.675724575 +0000 UTC m=+0.215504760 container create 76f50eb53076298c99fd061ac7e68082a5d718cef3bf294d9f43553a666f1c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:16:05.057 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:04.666+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:05.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:04.632+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:05.372 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:05 smithi019 podman[86428]: 2024-04-25 15:16:05.029736256 +0000 UTC m=+0.569516488 container init 76f50eb53076298c99fd061ac7e68082a5d718cef3bf294d9f43553a666f1c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T15:16:05.372 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:05 smithi019 podman[86428]: 2024-04-25 15:16:05.04143585 +0000 UTC m=+0.581216002 container start 76f50eb53076298c99fd061ac7e68082a5d718cef3bf294d9f43553a666f1c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:16:05.372 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:05 smithi019 podman[86428]: 2024-04-25 15:16:05.12809859 +0000 UTC m=+0.667878835 container attach 76f50eb53076298c99fd061ac7e68082a5d718cef3bf294d9f43553a666f1c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T15:16:05.763 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 15:16:05 smithi019 podman[86428]: 2024-04-25 15:16:05.51084819 +0000 UTC m=+1.050628382 container died 76f50eb53076298c99fd061ac7e68082a5d718cef3bf294d9f43553a666f1c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T15:16:05.790 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:05 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:05.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:06.057 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:05.698+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:06.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:05.634+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:06.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:05.761+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:06.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:05 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:05.834+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:06.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T15:16:06.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:06.423 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T15:16:06.423 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T15:16:06.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.4 2024-04-25T15:16:06.739 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:06 smithi083 systemd[1]: Stopping Ceph osd.4 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:06.740 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:06 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:06.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:07.041 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:06 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T15:16:06.737+0000 7f92832f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:07.041 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:06 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T15:16:06.737+0000 7f92832f5700 -1 osd.4 104 *** Got signal Terminated *** 2024-04-25T15:16:07.041 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:06 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T15:16:06.737+0000 7f92832f5700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:07.057 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:06.741+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:06.659+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:07.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:06 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:06.769+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:07.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:06 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:06.834+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:07.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:07 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:07.483+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:07.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:07 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:07.713+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:08.047 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:07.684+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:08.048 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:07.735+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:08.048 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:07.735+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:08.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:07 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:07.770+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:08.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:07 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:07.800+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:08.762 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:08 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:08.487+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:09.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:08 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:08.759+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:09.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:08.642+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:09.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:08.783+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:09.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:08.783+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:09.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:08 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:08.799+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:09.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:08 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:08.817+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:09.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:08 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:08.817+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:09.742 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:09 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:09.468+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:10.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:09 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:09.739+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:10.057 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:09.827+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:10.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:09.827+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:10.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:09.642+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:10.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:09.642+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:10.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:09 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:09.759+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:10.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:09 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:09.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:10.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:09 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:09.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:10.760 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:10 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:10.429+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:10.760 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:10 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T15:16:10.466+0000 7f927b8e1700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:41.261860+0000 front 2024-04-25T15:15:41.261877+0000 (oldest deadline 2024-04-25T15:16:10.062864+0000) 2024-04-25T15:16:10.845 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:10 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:10.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:10.845 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:10 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:10.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:10.846 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:10 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:10.579+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:10.871 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:10 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:10.800+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:10.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:10 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:10.800+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:10.872 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:10 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:10.787+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:10.872 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:10 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:10.787+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:11.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:10 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:10.758+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:11.309 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:10 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:10.844+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:11.310 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:10 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:10.844+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:11.775 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:11 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4[44451]: 2024-04-25T15:16:11.465+0000 7f927b8e1700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:41.261860+0000 front 2024-04-25T15:15:41.261877+0000 (oldest deadline 2024-04-25T15:16:10.062864+0000) 2024-04-25T15:16:11.776 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:11 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:11.392+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:11.776 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:11 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:11.392+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:11.828 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:11 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:11.566+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:11.829 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:11 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:11.598+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:11.829 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:11 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:11.598+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:12.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:11 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:11.774+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:12.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:11.746+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:12.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:11.747+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:12.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:11.841+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:12.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:11 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:11.841+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:12.307 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:11 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:11.827+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:12.308 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:11 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:11.827+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:12.365 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:12 smithi083 podman[68965]: 2024-04-25 15:16:12.080065407 +0000 UTC m=+5.465233864 container died c754c31279ba7489339624677095e1e38099b00e88625c3ab0d5682f920f7fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:16:12.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:12 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:12.401+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:12.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:12 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:12.401+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:12.724 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:12 smithi083 podman[68965]: 2024-04-25 15:16:12.480286688 +0000 UTC m=+5.865455136 container cleanup c754c31279ba7489339624677095e1e38099b00e88625c3ab0d5682f920f7fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T15:16:12.725 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:12 smithi083 bash[68965]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4 2024-04-25T15:16:12.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:12 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:12.574+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:12.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:12 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:12.787+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:12.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:12 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:12.787+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:12.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:12 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:12.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:12.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:12 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:12.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:13.041 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:12 smithi083 podman[68982]: 2024-04-25 15:16:12.723247085 +0000 UTC m=+0.645634896 container remove c754c31279ba7489339624677095e1e38099b00e88625c3ab0d5682f920f7fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T15:16:13.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:12 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:12.749+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:13.121 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:12.761+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:13.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:12.762+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:13.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:12.827+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:13.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:12 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:12.827+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:13.327 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:12.944584406 +0000 UTC m=+0.017880260 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:13.327 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:13.049014724 +0000 UTC m=+0.122310564 container create 9b855a04e207ab70f6a9380c61d305e9af83beb5901106c578d4e06bd35bceae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-25T15:16:13.327 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:13.251382612 +0000 UTC m=+0.324678450 container init 9b855a04e207ab70f6a9380c61d305e9af83beb5901106c578d4e06bd35bceae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-25T15:16:13.328 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:13.254361897 +0000 UTC m=+0.327657735 container start 9b855a04e207ab70f6a9380c61d305e9af83beb5901106c578d4e06bd35bceae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T15:16:13.328 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:13.326280158 +0000 UTC m=+0.399576010 container attach 9b855a04e207ab70f6a9380c61d305e9af83beb5901106c578d4e06bd35bceae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:16:13.754 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 25 15:16:13 smithi083 podman[69070]: 2024-04-25 15:16:13.554440432 +0000 UTC m=+0.627736272 container died 9b855a04e207ab70f6a9380c61d305e9af83beb5901106c578d4e06bd35bceae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T15:16:13.754 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:13 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:13.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:13.754 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:13 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:13.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:14.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:13 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:13.753+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:14.057 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:13 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:13.574+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:14.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:13 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:13.738+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:14.058 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:13 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:13.738+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:14.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:13 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:13.560+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:14.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:13 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:13.560+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:14.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:13.807+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:14.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:13.807+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:14.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:13.877+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:14.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:13.877+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:14.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T15:16:14.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:14.186 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T15:16:14.186 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T15:16:14.186 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.5 2024-04-25T15:16:14.387 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:14 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:14.385+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:14.387 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:14 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:14.385+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:14.483 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 systemd[1]: Stopping Ceph osd.5 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:14.756 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:14.584+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:14.756 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:14.547+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:14.756 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:14.547+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:14.757 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:14.482+0000 7f2b4a5f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:14.757 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:14.482+0000 7f2b4a5f9700 -1 osd.5 104 *** Got signal Terminated *** 2024-04-25T15:16:14.757 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:14.482+0000 7f2b4a5f9700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:14.757 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:14.755+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:14.757 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:14 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:14.755+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:15.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:14 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:14.720+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:15.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:14.792+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:15.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:14.792+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:15.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:14.898+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:15.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:14 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:14.898+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:15.693 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:15 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:15.415+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:15.693 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:15 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:15.415+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:15.693 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:15 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:15.415+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:15.811 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:15.608+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:15.811 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:15.608+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:15.812 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:15.755+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:15.812 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:15.755+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:15.812 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:15.541+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:15.812 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:15.541+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:15.813 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:15 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:15.541+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:16.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:15 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:15.691+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:16.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:15.780+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:16.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:15.780+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:16.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:15.918+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:16.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:15 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:15.918+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:16.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:16 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:16.694+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:16.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:16 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:16.461+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:16.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:16 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:16.461+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:16.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:16 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:16.462+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:16.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:16.727+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:16.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:16.727+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:16.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:16.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:16.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:16.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:16.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:16.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:16.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:16.611+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:16.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:16 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:16.611+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:17.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:16.769+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:17.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:16.769+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:17.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:16.769+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:17.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:16.915+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:17.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:16.915+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:17.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:16 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:16.915+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:17.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:17 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:17.672+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:17.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:17 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:17.478+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:17.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:17 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:17.478+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:17.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:17 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:17.478+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:17.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:17.691+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:17.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:17.691+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:17.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:17.691+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.824542+0000 front 2024-04-25T15:15:53.824589+0000 (oldest deadline 2024-04-25T15:16:17.324700+0000) 2024-04-25T15:16:17.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:17.655+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:17.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:17.655+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:17.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:17.495+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:17.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:17.495+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:17.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:17 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:17.495+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:18.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:17.753+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:18.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:17.753+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:18.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:17.753+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:18.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:17.921+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:18.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:17.921+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:18.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:17.921+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:18.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:18.705+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:18.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:18.705+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:18.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:18.520+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:18.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:18.520+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:18.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:18 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:18.520+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:18.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:18.683+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:18.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:18.683+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:18.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:18.678+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:34.022002+0000 front 2024-04-25T15:15:34.022349+0000 (oldest deadline 2024-04-25T15:15:59.921830+0000) 2024-04-25T15:16:18.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:18.678+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.323202+0000 front 2024-04-25T15:15:43.323379+0000 (oldest deadline 2024-04-25T15:16:07.423345+0000) 2024-04-25T15:16:18.808 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5[45109]: 2024-04-25T15:16:18.678+0000 7f2b46bed700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.824542+0000 front 2024-04-25T15:15:53.824589+0000 (oldest deadline 2024-04-25T15:16:17.324700+0000) 2024-04-25T15:16:18.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:18.485+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:18.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:18.485+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:18.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:18 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:18.485+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:19.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:18.772+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:19.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:18.772+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:19.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:18.772+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:19.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:18.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:19.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:18.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:19.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:18 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:18.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:19.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:19.706+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:19.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:19.706+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:19.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:19.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:19.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:19.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:19.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:19 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:19.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:19.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:19.662+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:19.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:19.662+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:19.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:19.480+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:19.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:19.480+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:19.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:19 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:19.480+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:20.124 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:19.776+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:20.124 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:19.776+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:20.125 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:19.776+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:20.125 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:19.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:20.125 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:19.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:20.125 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:19 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:19.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:20.217 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:19 smithi102 podman[70695]: 2024-04-25 15:16:19.904208758 +0000 UTC m=+5.530820437 container died 6c66127c306775fd7bf1f04982648cd1e8d7161819560ef276766acced8b1a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T15:16:20.469 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:20 smithi102 podman[70695]: 2024-04-25 15:16:20.307659119 +0000 UTC m=+5.934270799 container cleanup 6c66127c306775fd7bf1f04982648cd1e8d7161819560ef276766acced8b1a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:16:20.469 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:20 smithi102 bash[70695]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5 2024-04-25T15:16:20.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:20.689+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:20.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:20.689+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:20.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:20.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:20.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:20.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:20.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:20 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:20.486+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:20.807 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:20 smithi102 podman[70713]: 2024-04-25 15:16:20.525628786 +0000 UTC m=+0.614137657 container remove 6c66127c306775fd7bf1f04982648cd1e8d7161819560ef276766acced8b1a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:16:20.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:20.621+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:20.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:20.621+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:20.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:20.467+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:20.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:20.467+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:20.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:20 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:20.467+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:21.097 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:20 smithi102 podman[70806]: 2024-04-25 15:16:20.843251955 +0000 UTC m=+0.113107713 container create 70479893e5f8c9d4e188d3b43dd785ce1a62ac23c34ad96490b7877c1177d65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T15:16:21.097 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:20 smithi102 podman[70806]: 2024-04-25 15:16:20.748049041 +0000 UTC m=+0.017904816 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:21.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:20.822+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:21.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:20.822+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:21.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:20.822+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:21.125 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:20.840+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:21.126 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:20.840+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:21.126 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:20 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:20.840+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:21.389 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:21 smithi102 podman[70806]: 2024-04-25 15:16:21.09661519 +0000 UTC m=+0.366470949 container init 70479893e5f8c9d4e188d3b43dd785ce1a62ac23c34ad96490b7877c1177d65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:16:21.390 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:21 smithi102 podman[70806]: 2024-04-25 15:16:21.099795805 +0000 UTC m=+0.369651563 container start 70479893e5f8c9d4e188d3b43dd785ce1a62ac23c34ad96490b7877c1177d65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:16:21.390 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:21 smithi102 podman[70806]: 2024-04-25 15:16:21.187331901 +0000 UTC m=+0.457187679 container attach 70479893e5f8c9d4e188d3b43dd785ce1a62ac23c34ad96490b7877c1177d65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:16:21.682 INFO:journalctl@ceph.osd.5.smithi102.stdout:Apr 25 15:16:21 smithi102 podman[70806]: 2024-04-25 15:16:21.389134055 +0000 UTC m=+0.658989815 container died 70479893e5f8c9d4e188d3b43dd785ce1a62ac23c34ad96490b7877c1177d65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T15:16:21.683 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:21.595+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:21.683 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:21.595+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:21.684 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:21.450+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:21.684 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:21.450+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:21.684 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:21 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:21.450+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:21.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:21.723+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:21.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:21.723+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:21.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:21.479+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:21.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:21.479+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:21.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:21 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:21.479+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:21.990 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T15:16:22.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:22.029 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T15:16:22.029 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T15:16:22.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.6 2024-04-25T15:16:22.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:21.807+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:22.111 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:21.807+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:22.112 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:21.807+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:22.112 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:21.830+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:22.112 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:21.830+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:22.112 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:21 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:21.830+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:22.372 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 systemd[1]: Stopping Ceph osd.6 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:22.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:22.674+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:22.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:22.674+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:22.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:22.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:22.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:22.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:22.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:22 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:22.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:22.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:22.608+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:22.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:22.608+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:22.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:22.438+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:22.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:22.438+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:22.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:22 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:22.438+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:23.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:22.877+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:23.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:22.877+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:23.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:22.877+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.699+0000 7f2b73e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.699+0000 7f2b73e00700 -1 osd.6 104 *** Got signal Terminated *** 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.699+0000 7f2b73e00700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.803+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.803+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:23.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:22.803+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:23.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:23.664+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:23.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:23.664+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:23.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:23.475+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:23.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:23.475+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:23.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:23 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:23.475+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:23.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:23.562+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:23.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:23.562+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:23.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:23.427+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:23.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:23.428+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:23.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:23 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:23.428+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:24.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:23.766+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:24.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:23.766+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:24.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:23.766+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:24.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:23.858+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:24.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:23.858+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:24.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:23 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:23.858+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:24.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:24.669+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:24.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:24.669+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:24.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:24.485+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:24.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:24.485+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:24.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:24 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:24.485+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:24.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:24.539+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:24.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:24.539+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:24.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:24.439+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:24.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:24.439+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:24.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:24 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:24.439+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:25.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:24.815+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:25.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:24.815+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:25.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:24.815+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:25.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:24.867+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:25.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:24.867+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:25.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:24 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:24.867+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:25.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:25.653+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:25.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:25.653+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:25.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:25.653+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:25.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:25.440+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:25.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:25.440+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:25.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:25 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:25.440+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:25.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:25.472+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:25.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:25.472+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:25.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:25.472+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:25.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:25.527+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:25.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:25.527+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:25.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:25 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:25.527+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:26.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:25.857+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:26.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:25.857+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:26.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:25.857+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:26.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:25.863+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:26.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:25.863+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:26.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:25.863+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:26.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:25 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:25.863+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:26.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:26.692+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:26.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:26.692+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:26.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:26.692+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:26.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:26.451+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:26.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:26.451+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:26.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:26.451+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:26.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:26 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:26.451+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:26.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:26.471+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:26.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:26.471+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:26.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:26.471+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:26.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:26.554+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:26.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:26.554+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:26.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:26 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:26.554+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:27.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:26.830+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:32.818503+0000 front 2024-04-25T15:15:32.819043+0000 (oldest deadline 2024-04-25T15:15:58.718151+0000) 2024-04-25T15:16:27.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:26.830+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.422239+0000 front 2024-04-25T15:15:44.422208+0000 (oldest deadline 2024-04-25T15:16:10.322280+0000) 2024-04-25T15:16:27.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:26.830+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.725543+0000 front 2024-04-25T15:15:53.725378+0000 (oldest deadline 2024-04-25T15:16:16.025596+0000) 2024-04-25T15:16:27.123 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6[56401]: 2024-04-25T15:16:26.830+0000 7f2b6a600700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.228617+0000 front 2024-04-25T15:16:01.228799+0000 (oldest deadline 2024-04-25T15:16:26.430226+0000) 2024-04-25T15:16:27.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:26.824+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:27.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:26.824+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:27.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:26.824+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:27.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:26 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:26.824+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:27.709 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:27.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:27.709 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:27.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:27.709 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:27.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:27.709 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:27.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:27.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:27.456+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:27.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:27.456+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:27.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:27.456+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:27.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:27.558+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:27.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:27.558+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:27.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:27 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:27.558+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:28.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:27.708+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:28.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:27.708+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:28.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:27 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:27.708+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:28.109 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:27.791+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:28.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:27.791+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:28.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:27.791+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:28.110 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:27 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:27.791+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:28.371 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:28 smithi019 podman[86533]: 2024-04-25 15:16:28.108540478 +0000 UTC m=+5.595131046 container died bb54c6496b4a9a3a755d995ab31a657cc06604ed83b8e66d20eb91a387ab5be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T15:16:28.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:28.663+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:28.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:28.663+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:28.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:28.663+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:28.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:28.448+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:28.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:28.448+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:28.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:28.448+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:28.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:28 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:28.448+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:28.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:28.519+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:28.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:28.519+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:28.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:28.519+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:28.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:28.464+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:28.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:28.464+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:28.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:28 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:28.464+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:29.010 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:28 smithi019 podman[86533]: 2024-04-25 15:16:28.725435613 +0000 UTC m=+6.212026218 container cleanup bb54c6496b4a9a3a755d995ab31a657cc06604ed83b8e66d20eb91a387ab5be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:16:29.011 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:28 smithi019 bash[86533]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6 2024-04-25T15:16:29.011 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:28.792+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:29.011 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:28.792+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:29.011 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:28.792+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:29.011 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:28.792+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:29.372 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:29 smithi019 podman[86551]: 2024-04-25 15:16:29.008660382 +0000 UTC m=+0.891325971 container remove bb54c6496b4a9a3a755d995ab31a657cc06604ed83b8e66d20eb91a387ab5be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:16:29.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:29.675+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:29.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:29.675+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:29.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:29.676+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:29.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:29.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:29.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:29.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:29.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:29.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:29.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:29 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:29.442+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:29.486+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:29.486+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:29.486+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:29.440+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:29.440+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:29.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:29 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:29.440+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:30.119 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:29 smithi019 podman[86649]: 2024-04-25 15:16:29.798812623 +0000 UTC m=+0.040453416 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:30.119 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:30 smithi019 podman[86649]: 2024-04-25 15:16:30.118204825 +0000 UTC m=+0.359845639 container create 22419b1dde26c8dbd1598448669d17da84b65d326c336fb76cdec3eaae405d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T15:16:30.120 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:29.826+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:30.120 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:29.826+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:30.120 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:29.826+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:30.120 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:29 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:29.826+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:30.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:30.629+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:30.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:30.629+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:30.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:30.629+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:30.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:30.435+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:30.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:30.435+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:30.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:30.435+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:30.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:30 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:30.435+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:30.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:30.490+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:30.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:30.490+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:30.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:30.490+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:30.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:30.458+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:30.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:30.458+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:30.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:30 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:30.458+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:30.821 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:30 smithi019 podman[86649]: 2024-04-25 15:16:30.43012948 +0000 UTC m=+0.671770298 container init 22419b1dde26c8dbd1598448669d17da84b65d326c336fb76cdec3eaae405d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:16:30.822 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:30 smithi019 podman[86649]: 2024-04-25 15:16:30.442015271 +0000 UTC m=+0.683656089 container start 22419b1dde26c8dbd1598448669d17da84b65d326c336fb76cdec3eaae405d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T15:16:30.822 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:30 smithi019 podman[86649]: 2024-04-25 15:16:30.545442496 +0000 UTC m=+0.787083390 container attach 22419b1dde26c8dbd1598448669d17da84b65d326c336fb76cdec3eaae405d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T15:16:31.122 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 15:16:30 smithi019 podman[86649]: 2024-04-25 15:16:30.891340381 +0000 UTC m=+1.132981235 container died 22419b1dde26c8dbd1598448669d17da84b65d326c336fb76cdec3eaae405d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T15:16:31.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:30 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:30.819+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:31.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:30 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:30.819+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:31.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:30 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:30.819+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:31.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:30 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:30.819+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:31.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T15:16:31.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:31.778 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T15:16:31.778 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T15:16:31.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.7 2024-04-25T15:16:31.790 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:31.627+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:31.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:31.627+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:31.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:31.627+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:31.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:31.450+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:31.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:31.450+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:31.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:31.450+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:31.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:31 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:31.450+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:31.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:31.528+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:31.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:31.528+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:31.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:31.528+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:31.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:31.484+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:31.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:31.484+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:31.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:31 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:31.484+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:31.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:31.812+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:31.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:31.812+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:31.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:31.812+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:31.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:31 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:31.812+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:32.116 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:31 smithi083 systemd[1]: Stopping Ceph osd.7 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:32.399 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.115+0000 7f3c9008a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:32.400 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.115+0000 7f3c9008a700 -1 osd.7 104 *** Got signal Terminated *** 2024-04-25T15:16:32.400 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.115+0000 7f3c9008a700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:32.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:32.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:32.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:32.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:32.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:32.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:32.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:32.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:32.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.645+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:32.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.645+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:32.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.645+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:32.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:32 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:32.645+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:11.385109+0000 front 2024-04-25T15:16:11.385185+0000 (oldest deadline 2024-04-25T15:16:32.485224+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:32.539+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:32.539+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:32.539+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:32.490+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:32.490+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:32.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:32 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:32.490+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:33.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:32 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:32.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:33.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:32 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:32.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:33.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:32 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:32.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:33.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:32 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:32.811+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:33.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:33.416+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:33.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:33.416+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:33.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:33.416+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:33.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:33.416+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:33.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:33.599+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:33.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:33.599+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:33.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:33.599+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:33.792 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:33 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:33.599+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:11.385109+0000 front 2024-04-25T15:16:11.385185+0000 (oldest deadline 2024-04-25T15:16:32.485224+0000) 2024-04-25T15:16:33.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:33.553+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:33.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:33.553+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:33.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:33.553+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:33.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:33.476+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:33.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:33.476+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:33.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:33.476+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:33.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:33 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:33.476+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:34.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:33 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:33.803+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:34.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:33 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:33.803+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:34.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:33 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:33.803+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:34.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:33 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:33.803+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:34.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:33 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:33.803+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:34.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:34.561+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:34.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:34.561+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:34.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:34.561+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:34.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:34.561+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:11.385109+0000 front 2024-04-25T15:16:11.385185+0000 (oldest deadline 2024-04-25T15:16:32.485224+0000) 2024-04-25T15:16:34.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:34.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:34.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:34.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:34.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:34.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:34.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:34.443+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:34.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:34.574+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:34.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:34.574+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:34.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:34.574+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:34.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:34.510+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:34.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:34.510+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:34.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:34.510+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:34.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:34 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:34.510+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:35.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:34 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:34.850+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:35.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:34 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:34.850+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:35.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:34 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:34.850+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:35.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:34 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:34.850+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:35.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:34 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:34.850+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:35.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:35.521+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:35.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:35.521+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:35.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:35.521+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:35.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:35.521+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:11.385109+0000 front 2024-04-25T15:16:11.385185+0000 (oldest deadline 2024-04-25T15:16:32.485224+0000) 2024-04-25T15:16:35.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:35.471+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:35.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:35.471+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:35.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:35.471+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:35.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:35 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:35.471+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:35.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:35.540+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:35.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:35.540+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:35.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:35.540+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:35.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:35.540+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:35.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:35.533+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:35.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:35.533+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:35.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:35.533+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:35.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:35 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:35.533+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:36.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:35 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:35.821+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:36.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:35 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:35.821+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:36.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:35 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:35.821+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:36.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:35 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:35.821+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:36.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:35 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:35.821+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:36.561 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:36.505+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:36.561 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:36.505+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:36.505+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:36.505+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:36.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:36.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:36.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:36.562 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:36 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:36.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:36.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:36.551+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.979583+0000 front 2024-04-25T15:15:45.979606+0000 (oldest deadline 2024-04-25T15:16:07.580551+0000) 2024-04-25T15:16:36.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:36.551+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.981975+0000 front 2024-04-25T15:15:53.981886+0000 (oldest deadline 2024-04-25T15:16:18.081956+0000) 2024-04-25T15:16:36.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:36.551+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:15:58.082747+0000 front 2024-04-25T15:15:58.082685+0000 (oldest deadline 2024-04-25T15:16:25.083113+0000) 2024-04-25T15:16:36.791 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7[51309]: 2024-04-25T15:16:36.551+0000 7f3c88676700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:11.385109+0000 front 2024-04-25T15:16:11.385185+0000 (oldest deadline 2024-04-25T15:16:32.485224+0000) 2024-04-25T15:16:36.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:36.441+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:36.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:36.441+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:36.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:36.441+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:36.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:36.441+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:36.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:36.441+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:37.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:36 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:36.825+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:37.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:36 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:36.825+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:37.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:36 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:36.825+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:37.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:36 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:36.825+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:37.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:36 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:36.825+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:37.759 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:37 smithi083 podman[69187]: 2024-04-25 15:16:37.461198748 +0000 UTC m=+5.493094659 container died 194adac4c321eba063bbe8e818dffe6905e3d7e50bf58195860befb2215daf5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T15:16:37.759 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:37 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:37.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:37.759 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:37 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:37.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:37.760 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:37 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:37.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:37.760 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:37 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:37.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:37.760 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:37 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:37.405+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:37.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:37.599+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:37.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:37.599+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:37.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:37.599+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:37.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:37.599+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:37.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:37.526+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:37.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:37.526+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:37.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:37.526+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:37.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:37 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:37.526+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:38.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:37 smithi083 podman[69187]: 2024-04-25 15:16:37.83209403 +0000 UTC m=+5.863989937 container cleanup 194adac4c321eba063bbe8e818dffe6905e3d7e50bf58195860befb2215daf5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T15:16:38.041 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:37 smithi083 bash[69187]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7 2024-04-25T15:16:38.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:37 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:37.832+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:38.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:37 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:37.832+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:38.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:37 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:37.832+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:38.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:37 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:37.832+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:38.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:37 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:37.832+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:38.363 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69205]: 2024-04-25 15:16:38.083175806 +0000 UTC m=+0.616883909 container remove 194adac4c321eba063bbe8e818dffe6905e3d7e50bf58195860befb2215daf5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T15:16:38.637 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:38 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:38.362+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:38.637 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:38 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:38.362+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:38.637 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:38 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:38.362+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:38.637 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:38 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:38.362+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:38.637 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:38 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:38.362+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:38.638 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.28685824 +0000 UTC m=+0.018547789 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:38.638 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.39272231 +0000 UTC m=+0.124411922 container create 289fe5a7219f316b852d81c3ba48c11cd6d1632b7b6e18c4d57af46c7959b11d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:16:38.638 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.636634492 +0000 UTC m=+0.368324041 container init 289fe5a7219f316b852d81c3ba48c11cd6d1632b7b6e18c4d57af46c7959b11d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T15:16:38.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:38.487+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:38.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:38.487+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:38.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:38.487+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:38.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:38.487+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:38.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:38.557+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:38.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:38.557+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:38.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:38.557+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:38.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:38 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:38.557+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:38.932 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.640057445 +0000 UTC m=+0.371747002 container start 289fe5a7219f316b852d81c3ba48c11cd6d1632b7b6e18c4d57af46c7959b11d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T15:16:38.932 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.728279527 +0000 UTC m=+0.459969079 container attach 289fe5a7219f316b852d81c3ba48c11cd6d1632b7b6e18c4d57af46c7959b11d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T15:16:39.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:38 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:38.866+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:39.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:38 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:38.866+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:39.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:38 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:38.866+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:39.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:38 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:38.866+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:39.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:38 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:38.866+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:39.240 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 25 15:16:38 smithi083 podman[69302]: 2024-04-25 15:16:38.93151376 +0000 UTC m=+0.663203309 container died 289fe5a7219f316b852d81c3ba48c11cd6d1632b7b6e18c4d57af46c7959b11d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:16:39.515 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:39.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:39.516 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:39.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:39.516 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:39.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:39.516 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:39.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:39.516 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:39 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:39.411+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:39.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T15:16:39.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:39.628 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T15:16:39.629 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T15:16:39.629 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.8 2024-04-25T15:16:39.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:39.521+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:39.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:39.521+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:39.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:39.521+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:39.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:39.521+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:39.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.502+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:39.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.502+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:39.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.502+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:39.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.502+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:39.809 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 systemd[1]: Stopping Ceph osd.8 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:40.307 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.926+0000 7f5ef8d85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:40.308 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.926+0000 7f5ef8d85700 -1 osd.8 104 *** Got signal Terminated *** 2024-04-25T15:16:40.308 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:39 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:39.926+0000 7f5ef8d85700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:40.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:39 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:39.904+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:40.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:39 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:39.904+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:40.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:39 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:39.904+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:40.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:39 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:39.904+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:40.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:39 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:39.904+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:40.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:40.445+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:40.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:40.445+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:40.725 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:40.445+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:40.725 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:40.445+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:40.725 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:40.445+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:40.807 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:40.537+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:40.537+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:40.537+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:40.537+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:40.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:40.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:40.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:40.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:40.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:40 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:40.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:41.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:40 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:40.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:41.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:40 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:40.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:41.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:40 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:40.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:41.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:40 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:40.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:41.375 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:40 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:40.899+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:41.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:41.436+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:41.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:41.436+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:41.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:41.436+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:41.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:41.436+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:41.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:41 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:41.436+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:42.057 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:41.578+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:42.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:41.578+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:42.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:41.578+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:42.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:41.578+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:42.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:41.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:42.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:41.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:42.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:41.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:42.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:41.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:42.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:41 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:41.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:42.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:41 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:41.851+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:42.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:41 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:41.851+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:42.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:41 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:41.851+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:42.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:41 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:41.851+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:42.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:41 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:41.851+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:42.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:42 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:42.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:42.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:42 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:42.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:42.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:42 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:42.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:42.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:42 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:42.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:42.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:42 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:42.463+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:43.057 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:42.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:43.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:42.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:43.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:42.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:43.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:42.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:43.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:42.639+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:43.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:42.639+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:43.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:42.639+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:43.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:42.639+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:43.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:42 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:42.639+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:43.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:42.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:43.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:42.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:43.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:42.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:43.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:42.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:43.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:42 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:42.862+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:43.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:43.510+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:43.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:43.510+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:43.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:43.510+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:43.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:43.510+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:43.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:43 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:43.510+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:44.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:43.576+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:44.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:43.576+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:44.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:43.576+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:44.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:43.576+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:44.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:43.576+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:17.568883+0000 front 2024-04-25T15:16:17.568856+0000 (oldest deadline 2024-04-25T15:16:42.868881+0000) 2024-04-25T15:16:44.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:43.650+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:44.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:43.650+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:44.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:43.650+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:44.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:43.650+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:44.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:43 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:43.650+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:44.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:44.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:44.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:44.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:44.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:44.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:43 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:43.910+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:45.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:44.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:45.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:44.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:45.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:44.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:45.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:44.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:45.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:44.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:45.057 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:44.571+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:45.563884+0000 front 2024-04-25T15:15:45.564113+0000 (oldest deadline 2024-04-25T15:16:09.663922+0000) 2024-04-25T15:16:45.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:44.571+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:49.664339+0000 front 2024-04-25T15:15:49.664272+0000 (oldest deadline 2024-04-25T15:16:14.964408+0000) 2024-04-25T15:16:45.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:44.571+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.966129+0000 front 2024-04-25T15:16:01.966249+0000 (oldest deadline 2024-04-25T15:16:24.866086+0000) 2024-04-25T15:16:45.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:44.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.067669+0000 front 2024-04-25T15:16:10.067641+0000 (oldest deadline 2024-04-25T15:16:35.367630+0000) 2024-04-25T15:16:45.058 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8[52109]: 2024-04-25T15:16:44.572+0000 7f5ef1371700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:17.568883+0000 front 2024-04-25T15:16:17.568856+0000 (oldest deadline 2024-04-25T15:16:42.868881+0000) 2024-04-25T15:16:45.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:44.628+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:45.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:44.628+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:45.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:44.628+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:45.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:44.628+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:45.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:44 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:44.628+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:45.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:45.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:45.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:45.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:45.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:45.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:44 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:44.907+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:45.636 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:45 smithi102 podman[70912]: 2024-04-25 15:16:45.376793607 +0000 UTC m=+5.548581294 container died ff652537cd3b6dca9c66f628898ad00b12956e38939bebbb8a02d660791c997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T15:16:45.637 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:45 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:45.581+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:45.637 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:45 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:45.581+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:45.637 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:45 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:45.581+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:45.637 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:45 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:45.581+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:45.637 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:45 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:45.581+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:46.012 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:45 smithi102 podman[70912]: 2024-04-25 15:16:45.743419253 +0000 UTC m=+5.915206939 container cleanup ff652537cd3b6dca9c66f628898ad00b12956e38939bebbb8a02d660791c997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:16:46.012 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:45 smithi102 bash[70912]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:46.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:45 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:45.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:46.307 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[70929]: 2024-04-25 15:16:46.011217904 +0000 UTC m=+0.634081011 container remove ff652537cd3b6dca9c66f628898ad00b12956e38939bebbb8a02d660791c997a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T15:16:46.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:46.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:46.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:46.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:46.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:46.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:45 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:45.897+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:46.615 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.267170237 +0000 UTC m=+0.017864214 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:46.615 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.395496828 +0000 UTC m=+0.146190820 container create 66552ece0de75ca93eb7bc811f91a3114414a766c75c37533d52ca373f081e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T15:16:46.616 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:46 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:46.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:46.616 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:46 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:46.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:46.616 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:46 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:46.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:46.616 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:46 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:46.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:46.616 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:46 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:46.593+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:46.911 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.614726126 +0000 UTC m=+0.365420084 container init 66552ece0de75ca93eb7bc811f91a3114414a766c75c37533d52ca373f081e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:16:46.911 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.61842022 +0000 UTC m=+0.369114181 container start 66552ece0de75ca93eb7bc811f91a3114414a766c75c37533d52ca373f081e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T15:16:46.911 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.697726591 +0000 UTC m=+0.448420559 container attach 66552ece0de75ca93eb7bc811f91a3114414a766c75c37533d52ca373f081e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T15:16:47.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:47.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:47.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:47.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:47.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:47.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:46 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:46.551+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:47.192 INFO:journalctl@ceph.osd.8.smithi102.stdout:Apr 25 15:16:46 smithi102 podman[71019]: 2024-04-25 15:16:46.910685424 +0000 UTC m=+0.661379385 container died 66552ece0de75ca93eb7bc811f91a3114414a766c75c37533d52ca373f081e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T15:16:47.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:47.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:47.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:47.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:47.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:47.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:46 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:46.912+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:47.475 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T15:16:47.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:47.515 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T15:16:47.515 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T15:16:47.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.9 2024-04-25T15:16:47.724 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:47 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:47.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:47.724 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:47 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:47.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:47.724 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:47 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:47.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:47.724 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:47 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:47.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:47.725 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:47 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:47.605+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:47.872 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 systemd[1]: Stopping Ceph osd.9 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:48.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:48.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:48.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:48.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:48.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:48.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:47 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:47.560+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:48.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:48.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:48.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:48.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:48.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:48.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:47 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:47.891+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:48.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.099+0000 7f83c2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:48.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.099+0000 7f83c2400700 -1 osd.9 104 *** Got signal Terminated *** 2024-04-25T15:16:48.374 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.099+0000 7f83c2400700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:49.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:49.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:49.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:49.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:49.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:49.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:48 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:48.605+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:49.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:48 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:48.655+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:49.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:49.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:49.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:49.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:49.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:49.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:49.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:48 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:48.871+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.454409+0000 front 2024-04-25T15:16:26.454293+0000 (oldest deadline 2024-04-25T15:16:48.754007+0000) 2024-04-25T15:16:49.959 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:49.959 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:49.959 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:49.959 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:49.960 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:49.960 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:49.643+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:50.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:50.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:50.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:50.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:50.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:50.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:49 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:49.579+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:50.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:50.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:50.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:50.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:50.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:50.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:50.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:49 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:49.908+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.454409+0000 front 2024-04-25T15:16:26.454293+0000 (oldest deadline 2024-04-25T15:16:48.754007+0000) 2024-04-25T15:16:51.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:51.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:51.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:51.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:51.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:51.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:50 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:50.559+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:51.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:51.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:51.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:51.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:51.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:51.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:50 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:50.647+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:51.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.945+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:51.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.945+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:51.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.945+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:51.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.945+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:51.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.946+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:51.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.946+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:51.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:50 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:50.946+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.454409+0000 front 2024-04-25T15:16:26.454293+0000 (oldest deadline 2024-04-25T15:16:48.754007+0000) 2024-04-25T15:16:51.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:51.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:51.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:51.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:51.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:51.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:51.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:51 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:51.514+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:52.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:52.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:52.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:52.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:52.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:52.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:51 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:51.669+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:52.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:52.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:52.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:52.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:52.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:52.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:52.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:51 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:51.994+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.454409+0000 front 2024-04-25T15:16:26.454293+0000 (oldest deadline 2024-04-25T15:16:48.754007+0000) 2024-04-25T15:16:52.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:52.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:52.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:52.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:52.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:52.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:52.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:52 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:52.515+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:53.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:53.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:53.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:53.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:53.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:53.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:52 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:52.633+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:53.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.833567+0000 front 2024-04-25T15:15:36.833673+0000 (oldest deadline 2024-04-25T15:15:58.533231+0000) 2024-04-25T15:16:53.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:42.338348+0000 front 2024-04-25T15:15:42.338191+0000 (oldest deadline 2024-04-25T15:16:08.238153+0000) 2024-04-25T15:16:53.372 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.041684+0000 front 2024-04-25T15:15:54.041619+0000 (oldest deadline 2024-04-25T15:16:16.243238+0000) 2024-04-25T15:16:53.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:00.945825+0000 front 2024-04-25T15:16:00.946107+0000 (oldest deadline 2024-04-25T15:16:25.045228+0000) 2024-04-25T15:16:53.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:09.048244+0000 front 2024-04-25T15:16:09.048127+0000 (oldest deadline 2024-04-25T15:16:33.748333+0000) 2024-04-25T15:16:53.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:18.951403+0000 front 2024-04-25T15:16:18.951320+0000 (oldest deadline 2024-04-25T15:16:43.051561+0000) 2024-04-25T15:16:53.373 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9[62978]: 2024-04-25T15:16:53.033+0000 7f83bdc00700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.454409+0000 front 2024-04-25T15:16:26.454293+0000 (oldest deadline 2024-04-25T15:16:48.754007+0000) 2024-04-25T15:16:53.781 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 podman[86753]: 2024-04-25 15:16:53.516282029 +0000 UTC m=+5.554516439 container died b5e9fcdf62a83d3270ddb305f26e24e6f64859b78ba23fbe5d5720ef63b56ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:16:54.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:54.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:54.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:54.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:54.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:54.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:54.043 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:53 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:53.563+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:54.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:54.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:54.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:54.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:54.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:54.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:53.610+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:54.094 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 podman[86753]: 2024-04-25 15:16:53.900552976 +0000 UTC m=+5.938787369 container cleanup b5e9fcdf62a83d3270ddb305f26e24e6f64859b78ba23fbe5d5720ef63b56ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True) 2024-04-25T15:16:54.095 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:53 smithi019 bash[86753]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9 2024-04-25T15:16:54.371 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:54 smithi019 podman[86774]: 2024-04-25 15:16:54.09299933 +0000 UTC m=+0.569279524 container remove b5e9fcdf62a83d3270ddb305f26e24e6f64859b78ba23fbe5d5720ef63b56ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:16:55.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:55.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:55.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:55.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.610+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:55.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.610+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:55.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.610+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:55.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:54 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:54.610+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:55.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:55.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:55.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:55.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:55.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:55.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:54 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:54.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:55.306 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:54 smithi019 podman[86862]: 2024-04-25 15:16:54.815507572 +0000 UTC m=+0.036500890 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:16:55.306 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:54 smithi019 podman[86862]: 2024-04-25 15:16:54.952252219 +0000 UTC m=+0.173245577 container create 0393dc0b7d70797747f033aaf8aee66f14face9d106a69c1299be8037a4c73d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:16:55.622 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:55 smithi019 podman[86862]: 2024-04-25 15:16:55.305080088 +0000 UTC m=+0.526073419 container init 0393dc0b7d70797747f033aaf8aee66f14face9d106a69c1299be8037a4c73d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T15:16:55.622 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:55 smithi019 podman[86862]: 2024-04-25 15:16:55.316765494 +0000 UTC m=+0.537758847 container start 0393dc0b7d70797747f033aaf8aee66f14face9d106a69c1299be8037a4c73d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T15:16:55.622 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:55 smithi019 podman[86862]: 2024-04-25 15:16:55.387493146 +0000 UTC m=+0.608486482 container attach 0393dc0b7d70797747f033aaf8aee66f14face9d106a69c1299be8037a4c73d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T15:16:55.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:55.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:55.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:55.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:55.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:55.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:55 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:55.533+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:56.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:56.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:55 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:55.609+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:56.122 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 15:16:55 smithi019 podman[86862]: 2024-04-25 15:16:55.778104356 +0000 UTC m=+0.999097747 container died 0393dc0b7d70797747f033aaf8aee66f14face9d106a69c1299be8037a4c73d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T15:16:56.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T15:16:56.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:16:56.694 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T15:16:56.694 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T15:16:56.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.10 2024-04-25T15:16:57.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:57.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:57.004 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:57.004 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:56.641+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:57.004 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:56 smithi083 systemd[1]: Stopping Ceph osd.10 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:16:57.004 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.000+0000 7f2805611700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:16:57.004 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.000+0000 7f2805611700 -1 osd.10 104 *** Got signal Terminated *** 2024-04-25T15:16:57.005 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.000+0000 7f2805611700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:16:57.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:57.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:57.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:57.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:57.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:57.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:56 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:56.575+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:58.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:58.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:58.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:58.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:58.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:58.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:58.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:57 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:57.630+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:58.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:58.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:58.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:58.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:58.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:58.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:57 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:57.587+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:58.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:58.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:58.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:58.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:58.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:58.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:58 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:58.548+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:59.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:16:59.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:16:59.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:16:59.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:16:59.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:16:59.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:16:59.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:58 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:58.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:16:59.807 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:16:59.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:16:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:16:59.536+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:00.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:17:00.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:17:00.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:17:00.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:17:00.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:17:00.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:17:00.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:16:59 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:16:59.638+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:00.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:00.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:00 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:00.494+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:17:01.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:17:01.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:00 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:00.670+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:01.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:01 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:01.534+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:02.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:36.807154+0000 front 2024-04-25T15:15:36.807381+0000 (oldest deadline 2024-04-25T15:15:58.407406+0000) 2024-04-25T15:17:02.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:44.808992+0000 front 2024-04-25T15:15:44.809044+0000 (oldest deadline 2024-04-25T15:16:10.709198+0000) 2024-04-25T15:17:02.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:54.210572+0000 front 2024-04-25T15:15:54.210599+0000 (oldest deadline 2024-04-25T15:16:15.310321+0000) 2024-04-25T15:17:02.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T15:16:01.712548+0000 front 2024-04-25T15:16:01.712265+0000 (oldest deadline 2024-04-25T15:16:25.712612+0000) 2024-04-25T15:17:02.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.715057+0000 front 2024-04-25T15:16:10.715039+0000 (oldest deadline 2024-04-25T15:16:36.015105+0000) 2024-04-25T15:17:02.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.015739+0000 front 2024-04-25T15:16:16.015703+0000 (oldest deadline 2024-04-25T15:16:45.415948+0000) 2024-04-25T15:17:02.042 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:01 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10[57927]: 2024-04-25T15:17:01.687+0000 7f2801c05700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:25.416798+0000 front 2024-04-25T15:16:25.416656+0000 (oldest deadline 2024-04-25T15:16:50.617065+0000) 2024-04-25T15:17:02.660 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:02 smithi083 podman[69422]: 2024-04-25 15:17:02.382457412 +0000 UTC m=+5.497093552 container died a48d9febf89994fff86de4ffed6ead3afecb3078b71fa053080effdd4c9d48fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:02.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:02.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:02 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:02.523+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:03.010 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:02 smithi083 podman[69422]: 2024-04-25 15:17:02.758890583 +0000 UTC m=+5.873526718 container cleanup a48d9febf89994fff86de4ffed6ead3afecb3078b71fa053080effdd4c9d48fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T15:17:03.010 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:02 smithi083 bash[69422]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10 2024-04-25T15:17:03.290 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69441]: 2024-04-25 15:17:03.009855481 +0000 UTC m=+0.618518942 container remove a48d9febf89994fff86de4ffed6ead3afecb3078b71fa053080effdd4c9d48fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T15:17:03.631 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.228089531 +0000 UTC m=+0.017722442 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:17:03.631 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.327438595 +0000 UTC m=+0.117071500 container create 080cb61682dcd1820de93fa2157c1c7b7143b20d67c0e4927cdd347918d79b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:17:03.631 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.546447504 +0000 UTC m=+0.336080402 container init 080cb61682dcd1820de93fa2157c1c7b7143b20d67c0e4927cdd347918d79b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T15:17:03.631 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.549403216 +0000 UTC m=+0.339036115 container start 080cb61682dcd1820de93fa2157c1c7b7143b20d67c0e4927cdd347918d79b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T15:17:03.632 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.629537373 +0000 UTC m=+0.419170282 container attach 080cb61682dcd1820de93fa2157c1c7b7143b20d67c0e4927cdd347918d79b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T15:17:04.041 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 25 15:17:03 smithi083 podman[69528]: 2024-04-25 15:17:03.843587619 +0000 UTC m=+0.633220518 container died 080cb61682dcd1820de93fa2157c1c7b7143b20d67c0e4927cdd347918d79b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T15:17:04.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:04.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:03.572+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:04.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T15:17:04.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:04.489 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T15:17:04.489 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T15:17:04.489 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.11 2024-04-25T15:17:04.779 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 systemd[1]: Stopping Ceph osd.11 for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:04.780 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.544+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:04.781 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.777+0000 7f16dc2c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:04.781 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.777+0000 7f16dc2c7700 -1 osd.11 104 *** Got signal Terminated *** 2024-04-25T15:17:04.781 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:04 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:04.777+0000 7f16dc2c7700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:17:05.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:05.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:05.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:05.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:05.808 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:05.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:05.809 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:05 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:05.549+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:07.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:07.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:07.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:06 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:06.558+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-04-25T15:16:44.559297+0000 front 2024-04-25T15:16:44.559279+0000 (oldest deadline 2024-04-25T15:17:05.659358+0000) 2024-04-25T15:17:08.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:08.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:08.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:07 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:07.592+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-04-25T15:16:44.559297+0000 front 2024-04-25T15:16:44.559279+0000 (oldest deadline 2024-04-25T15:17:05.659358+0000) 2024-04-25T15:17:09.057 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:09.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:09.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:08 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:08.626+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-04-25T15:16:44.559297+0000 front 2024-04-25T15:16:44.559279+0000 (oldest deadline 2024-04-25T15:17:05.659358+0000) 2024-04-25T15:17:10.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T15:15:35.248478+0000 front 2024-04-25T15:15:35.248554+0000 (oldest deadline 2024-04-25T15:15:57.548296+0000) 2024-04-25T15:17:10.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-25T15:15:43.950183+0000 front 2024-04-25T15:15:43.950044+0000 (oldest deadline 2024-04-25T15:16:09.250130+0000) 2024-04-25T15:17:10.058 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-04-25T15:15:53.351367+0000 front 2024-04-25T15:15:53.351133+0000 (oldest deadline 2024-04-25T15:16:15.051171+0000) 2024-04-25T15:17:10.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-25T15:16:10.953134+0000 front 2024-04-25T15:16:10.953191+0000 (oldest deadline 2024-04-25T15:16:32.653068+0000) 2024-04-25T15:17:10.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-04-25T15:16:16.654531+0000 front 2024-04-25T15:16:16.654563+0000 (oldest deadline 2024-04-25T15:16:41.354723+0000) 2024-04-25T15:17:10.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T15:16:26.056140+0000 front 2024-04-25T15:16:26.055913+0000 (oldest deadline 2024-04-25T15:16:48.355753+0000) 2024-04-25T15:17:10.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-25T15:16:36.458016+0000 front 2024-04-25T15:16:36.457979+0000 (oldest deadline 2024-04-25T15:16:58.757770+0000) 2024-04-25T15:17:10.059 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:09 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11[59161]: 2024-04-25T15:17:09.606+0000 7f16d48b3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-04-25T15:16:44.559297+0000 front 2024-04-25T15:16:44.559279+0000 (oldest deadline 2024-04-25T15:17:05.659358+0000) 2024-04-25T15:17:10.687 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:10 smithi102 podman[71137]: 2024-04-25 15:17:10.310540797 +0000 UTC m=+5.630807207 container died 208ee67e6ea2f0a38c5292e3422a6af6ef25756217d2605f0a5e01d392e12670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T15:17:11.038 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:10 smithi102 podman[71137]: 2024-04-25 15:17:10.777898489 +0000 UTC m=+6.098164899 container cleanup 208ee67e6ea2f0a38c5292e3422a6af6ef25756217d2605f0a5e01d392e12670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T15:17:11.038 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:10 smithi102 bash[71137]: ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11 2024-04-25T15:17:11.307 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71152]: 2024-04-25 15:17:11.037514631 +0000 UTC m=+0.726575213 container remove 208ee67e6ea2f0a38c5292e3422a6af6ef25756217d2605f0a5e01d392e12670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T15:17:11.607 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.256038859 +0000 UTC m=+0.018201781 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:17:11.608 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.363416824 +0000 UTC m=+0.125579781 container create 64e84b68da9e3491d769d9caeecd7ffe4b4d4ab30bf660fd6cdfbb93f0f636f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:17:11.608 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.607190455 +0000 UTC m=+0.369353367 container init 64e84b68da9e3491d769d9caeecd7ffe4b4d4ab30bf660fd6cdfbb93f0f636f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T15:17:11.899 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.610280103 +0000 UTC m=+0.372443021 container start 64e84b68da9e3491d769d9caeecd7ffe4b4d4ab30bf660fd6cdfbb93f0f636f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T15:17:11.899 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.698919991 +0000 UTC m=+0.461082904 container attach 64e84b68da9e3491d769d9caeecd7ffe4b4d4ab30bf660fd6cdfbb93f0f636f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T15:17:12.193 INFO:journalctl@ceph.osd.11.smithi102.stdout:Apr 25 15:17:11 smithi102 podman[71248]: 2024-04-25 15:17:11.898207698 +0000 UTC m=+0.660370614 container died 64e84b68da9e3491d769d9caeecd7ffe4b4d4ab30bf660fd6cdfbb93f0f636f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T15:17:12.479 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T15:17:12.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:12.518 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T15:17:12.518 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T15:17:12.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.a 2024-04-25T15:17:12.872 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:12 smithi019 systemd[1]: Stopping Ceph mds.a for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:13.622 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: 2024-04-25T15:17:13.141+0000 7f28b4a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:13.622 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:13 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a[69463]: 2024-04-25T15:17:13.141+0000 7f28b4a00700 -1 mds.a *** got signal Terminated *** 2024-04-25T15:17:15.531 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:15 smithi019 podman[86992]: 2024-04-25 15:17:15.215802134 +0000 UTC m=+2.212375990 container died 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:17:15.872 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:15 smithi019 podman[86992]: 2024-04-25 15:17:15.677359569 +0000 UTC m=+2.673933421 container cleanup 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:17:15.872 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:15 smithi019 bash[86992]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a 2024-04-25T15:17:16.372 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 15:17:15 smithi019 podman[87006]: 2024-04-25 15:17:15.918389673 +0000 UTC m=+0.683375854 container remove 6383a965d87386e6d52dd4954c4e30abd0f9e709d2507ffee4366f056833d34d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T15:17:16.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T15:17:16.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:16.551 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T15:17:16.552 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T15:17:16.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.d 2024-04-25T15:17:17.122 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:16 smithi019 systemd[1]: Stopping Ceph mds.d for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:17.622 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d[68496]: 2024-04-25T15:17:17.253+0000 7feb33800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:17.622 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:17 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d[68496]: 2024-04-25T15:17:17.253+0000 7feb33800700 -1 mds.d *** got signal Terminated *** 2024-04-25T15:17:20.744 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:20 smithi019 podman[87112]: 2024-04-25 15:17:20.45762727 +0000 UTC m=+3.455290319 container died da177b9db64f82403bb49a2dee9a3fb416a707b1ca170762b71664e6b8f75bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T15:17:21.073 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:20 smithi019 podman[87112]: 2024-04-25 15:17:20.90441558 +0000 UTC m=+3.902078640 container cleanup da177b9db64f82403bb49a2dee9a3fb416a707b1ca170762b71664e6b8f75bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T15:17:21.073 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:20 smithi019 bash[87112]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d 2024-04-25T15:17:21.371 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 15:17:21 smithi019 podman[87128]: 2024-04-25 15:17:21.071717158 +0000 UTC m=+0.592833585 container remove da177b9db64f82403bb49a2dee9a3fb416a707b1ca170762b71664e6b8f75bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T15:17:21.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T15:17:21.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:21.720 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T15:17:21.720 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T15:17:21.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.g 2024-04-25T15:17:22.121 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:21 smithi019 systemd[1]: Stopping Ceph mds.g for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:22.622 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g[69111]: 2024-04-25T15:17:22.306+0000 7f71a8c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:22.622 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:22 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g[69111]: 2024-04-25T15:17:22.306+0000 7f71a8c00700 -1 mds.g *** got signal Terminated *** 2024-04-25T15:17:26.621 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:26 smithi019 podman[87233]: 2024-04-25 15:17:26.334014841 +0000 UTC m=+4.159564849 container died 01c29c231b62406050fe757fe1e3af7c610bfaa795e475a2b2ce6a810961fddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T15:17:27.122 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:26 smithi019 podman[87233]: 2024-04-25 15:17:26.890568304 +0000 UTC m=+4.716118305 container cleanup 01c29c231b62406050fe757fe1e3af7c610bfaa795e475a2b2ce6a810961fddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T15:17:27.122 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:26 smithi019 bash[87233]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g 2024-04-25T15:17:27.622 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 15:17:27 smithi019 podman[87246]: 2024-04-25 15:17:27.200028092 +0000 UTC m=+0.856487441 container remove 01c29c231b62406050fe757fe1e3af7c610bfaa795e475a2b2ce6a810961fddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T15:17:27.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T15:17:27.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:27.854 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T15:17:27.854 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T15:17:27.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.j 2024-04-25T15:17:28.122 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:27 smithi019 systemd[1]: Stopping Ceph mds.j for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:28.871 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j[68121]: 2024-04-25T15:17:28.433+0000 7fe17b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:28.872 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:28 smithi019 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j[68121]: 2024-04-25T15:17:28.434+0000 7fe17b200700 -1 mds.j *** got signal Terminated *** 2024-04-25T15:17:32.533 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:32 smithi019 podman[87356]: 2024-04-25 15:17:32.269582087 +0000 UTC m=+3.956492394 container died edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T15:17:32.872 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:32 smithi019 podman[87356]: 2024-04-25 15:17:32.695021292 +0000 UTC m=+4.381931619 container cleanup edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T15:17:32.872 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:32 smithi019 bash[87356]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j 2024-04-25T15:17:33.371 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 15:17:33 smithi019 podman[87382]: 2024-04-25 15:17:33.044878724 +0000 UTC m=+0.750656548 container remove edce94a2015bdbc92018588a3b0dffceede1c55a5545044d865d2f54b6421ef4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:17:33.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T15:17:33.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:33.703 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T15:17:33.703 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T15:17:33.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.b 2024-04-25T15:17:34.017 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:33 smithi083 systemd[1]: Stopping Ceph mds.b for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:34.277 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: 2024-04-25T15:17:34.016+0000 7efc18ae6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:34.278 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:34 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b[63157]: 2024-04-25T15:17:34.016+0000 7efc18ae6700 -1 mds.b *** got signal Terminated *** 2024-04-25T15:17:35.417 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:35 smithi083 podman[69632]: 2024-04-25 15:17:35.15454649 +0000 UTC m=+1.261561997 container died cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T15:17:35.784 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:35 smithi083 podman[69632]: 2024-04-25 15:17:35.5162015 +0000 UTC m=+1.623217007 container cleanup cbb6e2db6811f1c9eeca73092d9d86300c818cac7afbf74bb06efbfb0664b24a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:17:35.785 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 25 15:17:35 smithi083 bash[69632]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-b 2024-04-25T15:17:35.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T15:17:35.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:35.971 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T15:17:35.971 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T15:17:35.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.e 2024-04-25T15:17:36.541 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:36 smithi083 systemd[1]: Stopping Ceph mds.e for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:36.541 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e[62536]: 2024-04-25T15:17:36.295+0000 7f2d047ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:36.541 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:36 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e[62536]: 2024-04-25T15:17:36.295+0000 7f2d047ea700 -1 mds.e *** got signal Terminated *** 2024-04-25T15:17:39.471 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:39 smithi083 podman[69765]: 2024-04-25 15:17:39.145983068 +0000 UTC m=+2.959089678 container died 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T15:17:39.771 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:39 smithi083 podman[69765]: 2024-04-25 15:17:39.553328715 +0000 UTC m=+3.366435331 container cleanup 36772f42a56f2f0596379f9c7cc51e23a378a1e0f3bebfd0c0f249653c2c7851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T15:17:39.771 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 25 15:17:39 smithi083 bash[69765]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-e 2024-04-25T15:17:39.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T15:17:39.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:39.940 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T15:17:39.941 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T15:17:39.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.h 2024-04-25T15:17:40.040 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:40 smithi083 systemd[1]: Stopping Ceph mds.h for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:40.540 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h[62842]: 2024-04-25T15:17:40.264+0000 7f3438f76700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:40.541 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:40 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h[62842]: 2024-04-25T15:17:40.264+0000 7f3438f76700 -1 mds.h *** got signal Terminated *** 2024-04-25T15:17:43.699 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:43 smithi083 podman[69886]: 2024-04-25 15:17:43.429939123 +0000 UTC m=+3.274018372 container died d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T15:17:44.041 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:43 smithi083 podman[69886]: 2024-04-25 15:17:43.789847376 +0000 UTC m=+3.633926625 container cleanup d16373d2bcf08f984f2ef9346f5bb3f7d9406451f3acea5abd1c8fd9652357a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T15:17:44.041 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 25 15:17:43 smithi083 bash[69886]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-h 2024-04-25T15:17:44.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T15:17:44.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:44.245 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T15:17:44.245 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T15:17:44.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.k 2024-04-25T15:17:44.585 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:44 smithi083 systemd[1]: Stopping Ceph mds.k for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:44.706 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k[62217]: 2024-04-25T15:17:44.584+0000 7fdbe6c7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:44.706 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:44 smithi083 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k[62217]: 2024-04-25T15:17:44.584+0000 7fdbe6c7b700 -1 mds.k *** got signal Terminated *** 2024-04-25T15:17:48.394 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:48 smithi083 podman[70006]: 2024-04-25 15:17:48.108732418 +0000 UTC m=+3.649851879 container died 1b6a5c83e209104a589f0f0b455a2f1229f09b03c048a20e1d3ab7aa9995ae33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T15:17:48.727 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:48 smithi083 podman[70006]: 2024-04-25 15:17:48.476616007 +0000 UTC m=+4.017735453 container cleanup 1b6a5c83e209104a589f0f0b455a2f1229f09b03c048a20e1d3ab7aa9995ae33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T15:17:48.728 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 25 15:17:48 smithi083 bash[70006]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-k 2024-04-25T15:17:48.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T15:17:48.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:48.925 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T15:17:48.925 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T15:17:48.925 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.c 2024-04-25T15:17:49.294 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:48 smithi102 systemd[1]: Stopping Ceph mds.c for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:49.555 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c[64585]: 2024-04-25T15:17:49.292+0000 7f7187df8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:49.555 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:49 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c[64585]: 2024-04-25T15:17:49.292+0000 7f7187df8700 -1 mds.c *** got signal Terminated *** 2024-04-25T15:17:53.057 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:52 smithi102 podman[71354]: 2024-04-25 15:17:52.799143545 +0000 UTC m=+3.690020097 container died f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T15:17:53.369 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:53 smithi102 podman[71354]: 2024-04-25 15:17:53.126121691 +0000 UTC m=+4.016998236 container cleanup f1f64be28243da842fae824ad3ee45022ade5db4bbc5134f0ef4c6253d140686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T15:17:53.370 INFO:journalctl@ceph.mds.c.smithi102.stdout:Apr 25 15:17:53 smithi102 bash[71354]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-c 2024-04-25T15:17:53.594 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T15:17:53.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:53.629 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T15:17:53.629 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T15:17:53.630 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.f 2024-04-25T15:17:53.956 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:53 smithi102 systemd[1]: Stopping Ceph mds.f for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:54.307 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f[64275]: 2024-04-25T15:17:53.953+0000 7f9fd1162700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:54.308 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:53 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f[64275]: 2024-04-25T15:17:53.953+0000 7f9fd1162700 -1 mds.f *** got signal Terminated *** 2024-04-25T15:17:58.307 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:57 smithi102 podman[71488]: 2024-04-25 15:17:57.960379252 +0000 UTC m=+4.109323394 container died 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T15:17:58.690 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:58 smithi102 podman[71488]: 2024-04-25 15:17:58.43824311 +0000 UTC m=+4.587187264 container cleanup 0c032dc04d87aedd53ad60e2ba0537cdaf49fc2042b10f515917fb9aae5ef46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T15:17:58.691 INFO:journalctl@ceph.mds.f.smithi102.stdout:Apr 25 15:17:58 smithi102 bash[71488]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-f 2024-04-25T15:17:58.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T15:17:58.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:17:58.890 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T15:17:58.890 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T15:17:58.890 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.i 2024-04-25T15:17:59.202 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:17:58 smithi102 systemd[1]: Stopping Ceph mds.i for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:17:59.557 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:17:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i[64907]: 2024-04-25T15:17:59.201+0000 7f73c3d23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:17:59.558 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:17:59 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i[64907]: 2024-04-25T15:17:59.201+0000 7f73c3d23700 -1 mds.i *** got signal Terminated *** 2024-04-25T15:18:02.557 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:18:02 smithi102 podman[71608]: 2024-04-25 15:18:02.303864638 +0000 UTC m=+3.204333200 container died 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T15:18:02.927 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:18:02 smithi102 podman[71608]: 2024-04-25 15:18:02.660043263 +0000 UTC m=+3.560511825 container cleanup 170fde5d3b4526e2e47fedac0d04fa0813728cc759b503a808c99cbc39929331 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T15:18:02.928 INFO:journalctl@ceph.mds.i.smithi102.stdout:Apr 25 15:18:02 smithi102 bash[71608]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-i 2024-04-25T15:18:03.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T15:18:03.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:18:03.118 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T15:18:03.119 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T15:18:03.119 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.l 2024-04-25T15:18:03.438 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:03 smithi102 systemd[1]: Stopping Ceph mds.l for 42f950da-0313-11ef-bc93-c7b262605968... 2024-04-25T15:18:03.807 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l[63968]: 2024-04-25T15:18:03.437+0000 7f0949e76700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:18:03.807 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:03 smithi102 ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l[63968]: 2024-04-25T15:18:03.437+0000 7f0949e76700 -1 mds.l *** got signal Terminated *** 2024-04-25T15:18:05.431 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:05 smithi102 podman[71728]: 2024-04-25 15:18:05.179583178 +0000 UTC m=+1.849953856 container died 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T15:18:05.682 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:05 smithi102 podman[71728]: 2024-04-25 15:18:05.521693714 +0000 UTC m=+2.192064403 container cleanup 7d61b42d1a593a74fdd4cdcad4b29edb2ce7c99b34ed89404f9b05a3193621c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:18:05.682 INFO:journalctl@ceph.mds.l.smithi102.stdout:Apr 25 15:18:05 smithi102 bash[71728]: ceph-42f950da-0313-11ef-bc93-c7b262605968-mds-l 2024-04-25T15:18:05.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42f950da-0313-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T15:18:05.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:18:05.922 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T15:18:05.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:18:06.263 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:11.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:18:11.696 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:14.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:18:14.252 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:16.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:18:16.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:18:16.691 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:18:16.717 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T15:18:16.719 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi019/crash 2024-04-25T15:18:16.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:18:16.832 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:18:16.832 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:18:16.837 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi083/crash 2024-04-25T15:18:16.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:18:16.888 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:18:16.889 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:18:16.891 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi102/crash 2024-04-25T15:18:16.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:18:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/42f950da-0313-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:18:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:18:16.925 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T15:18:16.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T15:18:17.031 INFO:tasks.cephadm:Compressing logs... 2024-04-25T15:18:17.032 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:18:17.036 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:18:17.039 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:18:17.073 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T15:18:17.073 INFO:teuthology.orchestra.run.smithi102.stderr:: No such file or directory 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T15:18:17.074 INFO:teuthology.orchestra.run.smithi102.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.5.log: /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr: 88.1% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:18:17.075 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr: 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.8.log: 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T15:18:17.076 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T15:18:17.080 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T15:18:17.080 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T15:18:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T15:18:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-04-25T15:18:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log/var/log/ceph/cephadm.log: 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:18:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.10.log: 90.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T15:18:17.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:18:17.085 INFO:teuthology.orchestra.run.smithi083.stderr: 83.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:18:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T15:18:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T15:18:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T15:18:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T15:18:17.086 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T15:18:17.086 INFO:teuthology.orchestra.run.smithi102.stderr: 91.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T15:18:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T15:18:17.093 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T15:18:17.093 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:18:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T15:18:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T15:18:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T15:18:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T15:18:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T15:18:17.098 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:18:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T15:18:17.099 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:18:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T15:18:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T15:18:17.102 INFO:teuthology.orchestra.run.smithi019.stderr: 94.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T15:18:17.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T15:18:17.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:18:17.104 INFO:teuthology.orchestra.run.smithi019.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:18:17.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T15:18:17.107 INFO:teuthology.orchestra.run.smithi019.stderr: 94.3% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T15:18:17.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:18:17.107 INFO:teuthology.orchestra.run.smithi019.stderr: 83.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:18:17.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T15:18:17.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T15:18:17.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.9.log: 90.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-volume.log.gz 94.4% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T15:18:17.127 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T15:18:17.169 INFO:teuthology.orchestra.run.smithi019.stderr: 94.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T15:18:17.250 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T15:18:17.404 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T15:18:17.409 INFO:teuthology.orchestra.run.smithi102.stderr: 93.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T15:18:17.413 INFO:teuthology.orchestra.run.smithi102.stderr: 91.8% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T15:18:17.446 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T15:18:17.485 INFO:teuthology.orchestra.run.smithi102.stderr: 92.8% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T15:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr: 93.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T15:18:17.576 INFO:teuthology.orchestra.run.smithi083.stderr: 92.6% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T15:18:17.614 INFO:teuthology.orchestra.run.smithi102.stderr: 92.6% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T15:18:17.650 INFO:teuthology.orchestra.run.smithi019.stderr: 93.0% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T15:18:17.664 INFO:teuthology.orchestra.run.smithi019.stderr: 92.6% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T15:18:17.671 INFO:teuthology.orchestra.run.smithi019.stderr: 92.8% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T15:18:17.679 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T15:18:17.679 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T15:18:17.681 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T15:18:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m0.629s 2024-04-25T15:18:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m2.698s 2024-04-25T15:18:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.132s 2024-04-25T15:18:17.732 INFO:teuthology.orchestra.run.smithi102.stderr: 92.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T15:18:17.734 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-25T15:18:17.734 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m0.681s 2024-04-25T15:18:17.734 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m2.301s 2024-04-25T15:18:17.734 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.101s 2024-04-25T15:18:17.841 INFO:teuthology.orchestra.run.smithi019.stderr: 92.9% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T15:18:18.139 INFO:teuthology.orchestra.run.smithi019.stderr: 90.5% -- replaced with /var/log/ceph/42f950da-0313-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T15:18:18.143 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T15:18:18.144 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.075s 2024-04-25T15:18:18.144 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m3.592s 2024-04-25T15:18:18.144 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.349s 2024-04-25T15:18:18.145 INFO:tasks.cephadm:Archiving logs... 2024-04-25T15:18:18.149 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi019/log 2024-04-25T15:18:18.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:18:18.716 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi083/log 2024-04-25T15:18:18.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:18:19.105 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi102/log 2024-04-25T15:18:19.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:18:19.518 INFO:tasks.cephadm:Removing cluster... 2024-04-25T15:18:19.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force 2024-04-25T15:18:19.838 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:20.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force 2024-04-25T15:18:20.663 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:20.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42f950da-0313-11ef-bc93-c7b262605968 --force 2024-04-25T15:18:21.119 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 42f950da-0313-11ef-bc93-c7b262605968 2024-04-25T15:18:21.371 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T15:18:21.371 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:18:21.433 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:18:21.450 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:18:21.465 INFO:tasks.cephadm:Teardown complete 2024-04-25T15:18:21.465 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T15:18:21.477 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T15:18:21.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:18:21.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:18:21.492 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:18:21.654 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T15:18:21.672 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T15:18:21.672 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:18:21.680 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:18:21.683 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:18:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:18:21.697 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:18:21.700 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:18:21.701 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-25T15:18:21.701 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 7 377 2 +919us[ +958us] +/- 48ms 2024-04-25T15:18:21.701 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 129 -1339us[-1301us] +/- 67ms 2024-04-25T15:18:21.701 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -79us[ -79us] +/- 63ms 2024-04-25T15:18:21.702 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +934us[ +922us] +/- 48ms 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -1330us[-1341us] +/- 68ms 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -35us[ -48us] +/- 63ms 2024-04-25T15:18:21.703 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:18:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:18:21.742 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:18:21.742 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T15:18:21.743 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +874us[ +909us] +/- 47ms 2024-04-25T15:18:21.743 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -1370us[-1370us] +/- 69ms 2024-04-25T15:18:21.743 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -63us[ -63us] +/- 62ms 2024-04-25T15:18:21.743 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:18:21.745 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T15:18:21.759 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T15:18:21.760 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T15:18:21.771 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:18:21.970 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:18:22.041 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:18:22.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:18:22.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:22.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:18:22.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:22.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:18:22.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:22.265 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T15:18:22.279 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T15:18:22.293 INFO:teuthology.task.internal:Duration was 2067.711022 seconds 2024-04-25T15:18:22.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T15:18:22.305 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T15:18:22.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:18:22.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:18:22.315 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:18:22.351 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:18:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:18:22.444 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:18:22.849 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T15:18:22.849 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T15:18:22.849 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:18:22.935 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T15:18:22.936 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:18:22.963 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-04-25T15:18:22.963 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:18:22.984 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T15:18:22.984 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:18:22.992 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:18:23.006 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:18:23.079 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T15:18:23.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:18:23.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:18:23.087 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:18:23.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T15:18:23.631 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T15:18:23.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:18:23.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:18:23.746 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:18:23.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T15:18:23.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:23.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:23.792 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:23.813 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-25T15:18:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-04-25T15:18:23.888 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-25T15:18:23.932 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:23.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:23.983 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:23.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:23.997 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:18:24.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:18:24.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T15:18:24.024 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T15:18:24.025 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi019 2024-04-25T15:18:24.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:18:24.134 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi083 2024-04-25T15:18:24.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:18:24.171 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511/remote/smithi102 2024-04-25T15:18:24.171 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:18:24.207 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T15:18:24.208 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:18:24.216 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:18:24.218 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:18:24.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T15:18:24.284 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T15:18:24.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T15:18:24.298 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T15:18:24.298 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:18:24.304 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:18:24.306 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:18:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:18 /home/ubuntu/cephtest 2024-04-25T15:18:24.320 INFO:teuthology.orchestra.run.smithi102.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:18 /home/ubuntu/cephtest 2024-04-25T15:18:24.355 INFO:teuthology.orchestra.run.smithi019.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:18 /home/ubuntu/cephtest 2024-04-25T15:18:24.359 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T15:18:24.373 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T15:18:24.399 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T15:18:24.427 DEBUG:teuthology.nuke:shortname: smithi019 2024-04-25T15:18:24.427 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:18:24.454 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-25T15:18:24.454 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:18:24.460 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.806273', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhmrATMvssx5OYIsjI1YlKVPqtp3z85O1VC1IHWKbIF1iB3JhX9OcNDXWsJD82fWvj26hBhKwPh7Jh3/P1pEjQ='} 2024-04-25T15:18:24.481 DEBUG:teuthology.nuke:shortname: smithi102 2024-04-25T15:18:24.481 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:18:24.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.804225', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBApkJcznsj8WnsOIJ8SVPpV7JNCtjdQODYCXrZAnLG84xAi1O0ghj/GK+p0bBV5RAYmlVrklAmLd1GB1U//t3RA='} 2024-04-25T15:18:24.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:35:19.807798', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAZA+gtfIGwyGI91oCGb3Byh26KYW3iDJJL9Jmgauyojp5NdDXjBcHsVdIM1RTSBz3p11BZnESuxL4IVXfU8FA='} 2024-04-25T15:18:24.521 INFO:teuthology.orchestra.console.smithi019:Power off 2024-04-25T15:18:24.522 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:18:24.547 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-04-25T15:18:24.547 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:24.569 INFO:teuthology.orchestra.console.smithi102:Power off 2024-04-25T15:18:24.569 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:18:24.585 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-25T15:18:24.585 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:18:24.596 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T15:18:24.599 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2024-04-25T15:18:24.599 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:24.609 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-25T15:18:24.609 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:24.623 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-04-25T15:18:24.633 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-25T15:18:28.597 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:28.624 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:28.635 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:28.708 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T15:18:28.735 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-04-25T15:18:28.744 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-25T15:18:32.710 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:32.736 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:32.745 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:32.824 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T15:18:32.847 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2024-04-25T15:18:32.859 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-25T15:18:32.948 INFO:teuthology.orchestra.console.smithi102:Power off completed 2024-04-25T15:18:32.960 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-25T15:18:36.826 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:18:36.937 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-04-25T15:18:37.038 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-04-25T15:18:37.144 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 2067.7110216617584 failure_reason: 'Command failed on smithi019 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=eb637484bd754558891242e4e7789d25 status: fail success: false 2024-04-25T15:18:37.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T15:18:37.235 INFO:teuthology.run:FAIL